Immersive Animation for Staging and Music Videos
Creating an Animated World for Live Music
How Photography, Painting, Music and AI Came Together in an Early Experiment in Immersive Concert Visuals
Music has always created pictures in my head. Long before I began experimenting with artificial intelligence, I was interested in the relationship between sound and image and in the ways photography, filmmaking, painting, lighting and movement could change the way we experience music. Being both a musician and a visual artist has always made those worlds feel connected to me rather than separate creative disciplines.
This particular experiment began with a band I was playing in and a simple idea. I wondered what would happen if I could create an entire visual world around the musicians rather than simply making a traditional music video. At the time, generative AI and AI animation were relatively primitive compared with the tools available today. Characters changed unexpectedly, faces wandered, motion could fall apart and images that looked beautiful as still frames could become completely unusable once they began to move.
That unpredictability was also part of what made the experiment interesting. Instead of asking the technology what it could make for me, I began wondering how I could combine it with things I had already created. I had my photography, my paintings, the music and, most importantly, the actual musicians. The technology became another tool that allowed me to explore what might happen when all of those things collided.
The result became a strange, colorful and constantly moving world populated by musicians, rabbits, skeletons, flowers, birds and creatures that seemed to have wandered out of one of my paintings.
An early experiment by Charles Maring combining original photography, painting, music and AI assisted animation to create an immersive visual world for a band.
The World Began With a Painting
The visual foundation for the film came from one of my paintings, The Inside Out. Rather than beginning with a generic animation style and asking artificial intelligence to imitate something that already existed, I wanted the film to grow from imagery that was already mine. The painting gave me a visual language to work from. Its lines, textures, colors, figures and almost dreamlike accumulation of imagery became a kind of DNA for the animated world.
I began experimenting with ways to carry those qualities beyond the canvas and into environments where characters could exist and eventually move. That distinction mattered to me because I wasn't particularly interested in asking AI to decide what the work should look like. I wanted to see whether the technology could help me expand something I had already created and allow a painting to become something much larger than the physical canvas.
The Inside Out by Charles Maring. The colors, textures, characters and visual language of the painting became the creative foundation for the animated world.
Beginning With Real People
The musicians were just as important as the painting. This wasn't a fictional band assembled with artificial intelligence. These were real people I knew because I was one of them. I photographed Dan O'Keefe, Jay More, Jeff Piscitelli and Robert Pallman in my studio, creating portraits that gave me strong visual references for each musician. For my own character, I used a self portrait I had made in the art loft.
Photography gave me something real to begin with. Each person had a face, an instrument, a posture and a presence. The challenge wasn't to reproduce those photographs perfectly. I wanted to preserve enough of each person's identity that they could enter a completely different visual world and still feel connected to the actual musicians.
That became one of the most interesting parts of the experiment. The farther I pushed the imagery toward illustration and fantasy, the more important those original human references became. There was something very natural to me about starting with photography and real people before allowing technology to take the imagery somewhere completely unexpected.
From Portraits to Characters
Once I had the musicians and the visual language of the painting, the experimentation became much more ambitious. I began bringing those two sources together and exploring what the band might look like if they actually existed inside this world. I wasn't trying to achieve photographic realism. In fact, realism would have defeated much of what interested me about the idea. I wanted the musicians to become characters while retaining enough of themselves to remain recognizable.
Faces could become illustrated, clothing could absorb patterns and textures, and instruments could exist alongside impossible landscapes, skeletons, rabbits, birds, flowers and other strange inhabitants. There was a balancing act involved because pushing the transformation too little simply looked like a photograph with an effect applied to it, while pushing it too far caused the connection to the actual musician to disappear. Finding the space between those extremes became part of the creative process.
My photographic likeness translated into the visual language of The Inside Out. The objective was to remain recognizable while becoming part of the painting.
Building a World Rather Than a Background
Once the characters began working, my thinking changed. I wasn't simply creating animated portraits anymore. I was beginning to build a place for them to inhabit, and that opened the door to much stranger ideas. Rabbits could become enormous. Skeletons could wander through gardens. Birds, flowers and human figures could coexist without explanation. Scale no longer needed to behave logically, and the environments could continually evolve around the musicians.
The painting had already given me permission to think that way because a canvas doesn't have to obey reality. Once I began treating animation with the same freedom, the possibilities expanded dramatically. The technology also introduced its own accidents. Some were useless, while others were fascinating and occasionally pushed the imagery somewhere I would not have reached through a conventional production process.
Once the visual world was established, skeletons, rabbits, flowers and imagined landscapes became part of its evolving vocabulary.
What became increasingly important was deciding what belonged. That lesson has stayed with me as AI has evolved. Technology can generate possibilities at extraordinary speed, but possibility is not the same thing as direction. Someone still has to recognize what serves the idea, what deserves to remain and what is simply interesting noise.
When the Band Became Part of the Painting
Eventually the distinction between portrait, painting and animation began disappearing. The musicians no longer looked as though they had simply been placed in front of an illustrated background. They began to feel as though they actually belonged inside it. For me, that was when the experiment became much more interesting because I wasn't simply applying a style to a photograph anymore. I was creating a world with its own visual logic and allowing real people to inhabit it.
An Early Experiment
Looking at this film now is interesting because artificial intelligence has evolved tremendously since I made it. Many of the technical obstacles I encountered would be approached very differently today. Character consistency has improved, motion is becoming more controllable, generated environments can be far more sophisticated and the ability to combine traditional production with emerging tools continues to expand at an incredible pace.
For that reason, I don't look at this film as a demonstration of what today's AI can do. I see it as a record of an early experiment. I was exploring these tools before I had any certainty about where they were going and before many of the workflows that are becoming commonplace today existed. I wasn't waiting for someone to establish the rules or tell me what the accepted process should be. I was simply trying to understand what might happen if photography, painting, music and filmmaking collided with a technology that was still finding its own language.
Some experiments failed spectacularly and others opened doors. That has always been part of the creative process for me, whether I am working with a camera, a musical instrument, paint or something entirely new.
What I See Differently Now
At the time, I thought I was making an experimental music video. Looking back, I think I was also beginning to explore something much larger. I can now imagine these kinds of images filling an enormous video wall behind a band while the musicians perform in front of them. The environments could change with individual songs. Characters from one sequence could disappear and return later in the concert. Color, motion and imagery could follow the emotional arc of the music rather than simply functioning as decoration behind the performers.
The same thinking could extend well beyond bands. Theater productions could create environments that transform around performers without constructing an entirely new physical set for every scene. Touring artists could develop a visual language that belongs specifically to an album or tour. Festivals, immersive installations, corporate productions and live events could build environments where the visual experience becomes part of the performance itself.
That is one of the things that fascinates me most when I look back at this experiment today. Something that began on a computer screen could eventually become architectural, surrounding an audience at a scale that is completely different from the original film.
Music Gives the Images Their Rhythm
Being a musician changes the way I think about this kind of work because I don't see the imagery as something that should simply play behind a band. It needs a relationship with the music. A jam band makes that especially interesting because the music isn't always trying to arrive at exactly the same place in exactly the same way. Songs can stretch, musicians respond to one another and a section that lasted two minutes during one performance might last five during another.
I think the visuals created for that environment could behave differently as well. Some sequences might be precisely synchronized, while others could loop, evolve or respond more loosely to what is happening onstage. The imagery could breathe with the performance rather than forcing the musicians to follow a predetermined film. At that point, the screen behind the band stops feeling like a backdrop and begins to feel more like another instrument in the performance.
The Human Part of an Artificial World
There is something about this project that feels even more relevant to me now than it did when I made it. The process began with people. I photographed my friends. I knew the musicians and understood the music because I was playing it with them. The visual language came from a painting I had made with my own hands. My experience in photography influenced the portraits, my experience as a filmmaker influenced the movement and editing, and my experience as a musician influenced the rhythm.
Artificial intelligence entered somewhere in the middle of all of that. It didn't originate the relationships, the painting, the photography, the music or the idea. It gave me a new way to bring those things together and explore possibilities that would have been extraordinarily difficult for me to create otherwise.
That distinction has become increasingly important to the way I think about technology and is closely connected to the ideas I explore in Human First, my speaking program about creativity, communication and remaining human in an increasingly AI assisted world. Technology can dramatically expand what one person is capable of creating, but the most interesting question for me isn't simply what the technology can do. I am much more interested in what a person can do with it.
Where This Could Go
Today I look at this early experiment and see possibilities that extend far beyond the original film. I can imagine creating immersive concert visuals for touring musicians where an entire visual world is developed around the identity of the artist and the music. That world might begin with paintings, photography, drawings, archival material, album artwork or completely original imagery. It could combine traditional filmmaking and animation with generative technology rather than relying entirely on any one technique.
I can imagine enormous video walls that change throughout a performance, theatrical environments that transform without physical scenery, album artwork that becomes a moving world, or visual sequences built specifically around individual songs. Because the creative process can begin long before the final production, artists and production teams can also see and experience the direction before it reaches the stage. Ideas can be developed, tested and refined rather than merely described.
The technology has changed enormously since I made this film, and I have changed along with it. What interests me now isn't recreating this experiment with better technology. It is discovering where the idea could go next and what might be possible when the tools available today are combined with photography, filmmaking, painting, music and a live performance on a much larger scale.
The Tools Change
I have spent most of my creative life moving between disciplines. Photography led me toward filmmaking. Music changed the way I understood rhythm and editing. Painting gave me another way to think about color, texture and abstraction. Emerging technology has given me new ways to bring those things together. I don't see those disciplines as separate careers anymore. Increasingly, they feel like different instruments available for telling a story or creating an experience.
Sometimes the right tool is a camera. Sometimes it is a guitar. Sometimes it is paint on a canvas. Sometimes it might be a cinema screen stretching across the back of a stage, filled with a world that began with a portrait, a painting and an idea. And sometimes it is a technology that barely existed a few years earlier.
The tools will continue to change, just as they always have. What interests me is what becomes possible when we stop thinking about the tools as the idea and start thinking about what we actually want to create with them.