After the Apocalypse - AI Art and Experiment
Over a month ago, I started creating digital art and practicing my art skills that gone in time. I am rediscovering my passion for drawing and painting as the last time I draw was at least two years ago, a banner for my cousin's wedding. Starting over is always hard and disappointing. I may not be as good as a master painter, but I do make decent arts before.
As I start practicing my sketching skills, I find the repetition of what I produce boring despite being fruitful to my skills. I may continue to produced art from my hand, but my day job limits me to do more hours of practice. But I was eager to join and create artwork again and join the digital illustrations and NFT trend. I find a way to produce art at a faster pace but still pleasing. I was fortunate to come across AI tools that can create art.
 |
 |
At a glance, we look at AI arts as an easy, one-click artwork, but we are wrong. When we create AI art, we need to experiment and understand how to make the output better. Sometimes it is about understanding how the model works. We also need to account for how we structure our prompts and post-processing the artwork to look better. As I shared previously, the structure of our prompts affects how the AI model generates our artwork. I have shared spelling accuracy, a prompt for the art style, and our choice of words changes how our illustrations materialize.
In the second round of experiments with VQGAN +CLIP, I tried to make artworks depicting a world after an apocalypse. I was expecting that I would see a post-apocalyptic world often exists in sci-fi books and movies. But I am lowering my expectation at the same time since I might see in the illustration just patches of the earthquake, hurricane, or flooding photos. Aside from the theme of my prompt consisting of apocalypse and concept art, I was trying to observe if the AI model replicates an output over a series of iteration of the same prompt.
I had run the experimentation over the last two weeks and repeating only one prompt. I was astonished that, throughout the experiments, I was able to see some of the output has an uncanny resemblance of elements in the illustration to the previous one but is still not a replica. For example, the artworks above. The first artwork seems a wide-angle view of the apocalyptic world. We can see five ruined buildings, but the later illustration is seemingly the future state of the former, where the building remains washed out over time. Those output illustrations are the succession when I run the experiment.
 |
 |
 |
As I reflected on what transpired, I was eager to replicate the result. I did run countless iterations to come up with the next set of artworks. The four output illustration evolves from the first image. It is as if the elements are a future state of the previous one. I still don't know why the result seems to produce the illustrations in succession that ostensibly related to the previous one. I speculate that maybe the successful running of one prompt allowing the AI model to predict what happens next or evolve from the prior result. It happens even without having to set the previous image as starting image. Is it amazing? Yes, it is.
Due to the result, to satisfy my curiosity, I did look up a scholarly article to at least understand what happened. I didn't fully get all facts from the papers, but VQGAN+CLIP and similar AI art used GAN and autoregressive models to synthesis the image despite not training the model. I presumed that as we used the model over and over again with the same prompt. It may have learned the prompt that enables it to generate a future state of it. But I am not still sure about that.
At first, I think the illustration below is just one of the outputs with patches of images that failed to form. As I took a closer look, I was in awe of what I realized. If the prior output is the past state and the new result is a future state. Then, it will tell a bigger story. The latest four artwork shows how the facades became a ruin and washed up in the water, but, accounting for the blurry white walls in the illustration below, it seems walls of a new city or metropolis (established after the apocalypse). Again, these are not scientific, and it may just be a coincidence.
 |
As concluding thoughts, AI has been used to predict or forecast future values. It may be speculative and coincidental that my AI artworks evolve from the former results. It may be because the model learns the prompt well or just a needle-on-a-haystack moment, but I guess there is more to the AI art and create masterpieces from a prompt. This round of experiments and creative pursuit exemplifies my previous statement. Art is something we make to understand and express ourselves, and may AI do too. Despite arguments of whether it is art or not. AI art is an art and an expression of our creativity and craziness, sometimes.
| Author's Note: |
|---|
The artwork shown here are the best result since some are just a tapestry of images. To present what has transpired accurately, I did not edit any details in the artwork except for enhancing the resolution, color, and tone using Real-ESGAN and Photoshop. All content including digital illustrations is from the author. The illustrations are generated by VQGAN+CLIP (Notebook 1, Notebook 2 and Nightcafe) |
| NFT Gallery |
|---|
Check out After the Apocalypse at OpenSea |
Super cool man kaayo 😻
Thank you! Have some !PIZZA while exploring hive.
https://twitter.com/juecoree/status/1445734115685199877
The rewards earned on this comment will go directly to the person sharing the post on Twitter as long as they are registered with @poshtoken. Sign up at https://hiveposh.com.
PIZZA Holders sent $PIZZA tips in this post's comments:
juecoree tipped castleberry (x1)
jotakrevs tipped juecoree (x1)
@juecoree(2/10) tipped @goldgrifin007 (x1)
juecoree tipped itsmiessyonpeakd (x1)
Learn more at https://hive.pizza.
Interested in doing my own versions of these. Can you make a guide how to?
The AI tries to figure out relations from previous parts relative to its position. Previous AI attempts meant just splattering new things while rendering without minding the previous images it rendered making the work more chaotic. Recall previous AI art attempts and you see these were more crude than what the newer versions are.
Yeah, I can. I'll share it with you.
I think that was when they rely heavily on synthesis using Machine Learning rather than Deep Learning framework. I think it started to become eloquent in creating synthesis when GAN (Generative adversarial network) was conceptualized.
One thing that a few of us experimenting found out is that the seed you use in the parameters matters for consistency. If you have your seed set to -1 (which most of the notebooks have on default) then you will get a different path when the AI does it's iterations. If you haven't already, try running your experiment using the exact same seed on your runs and see how that turns out. Thanks for such a thorough explanation of your approach and experiences.
I would also encourage you to continue your sketching and hand-made art as time permits. It's like riding a bicycle... you just got to hop back on and in time it comes back. I think you could also use your sketches with the AI to play with too!
Keep up the great work and thanks for sharing here with us!
I did set it to - 1. I was expecting a big difference since the AI start differently at every iteration. That is why I was amaze how it generate some resemblance to a prior output. I may try setting the seed to a positive integer and start with the same noise as AI will do the iteration.
Thanks for the encouragement. I will try to get back my old skills.
I might try it out. I think this would be fun. !PIZZA
I was going to keep track of the random seeds generated by the -1 starting point whenever i got cool results so i could experiment later... but i haven't been able to find time to play around much lately. I think keeping track of good seeds would be cool to have on record for potential use cases in future experiments. Thanks for the response and for the Pizza! =)
This is very interesting!
Your post opened many questions in my head. For what I know, the AI, in a same session, learns from every run that you are doing... that's why for a same prompt or very similar one, it keeps the same style in all the artworks. But never thought that the series of runs could be connected following some kind of timeline. The amazing is also, that you got such timelining in different sessions... when I start a new session, most of the times, the AI takes a particular style, different from another sessions, using even the same prompt in all of them. But that is not what happened to you, so I think should consider again my previous assumptions.
Thanks for share your experiments and thoughts with us!
Grab this !PIZZA!
In my first theme of prompt, I had the same observation (even running same prompt at first) not until my second round of experiments. When I run the same prompt over and over again, it produces an uncanny similarity, but I did not pay attention at first. When I did take a closer look at the images, I find it somehow related to the first, but with all the iteration, only this set (shown in the blog) has a recognizable timeline. For now, I think it is coincidental and maybe due to running countless loop for a the same prompt. Maybe, as we explore this AI models, we can find new peculiarity in the output.
Hi @juecoree, indeed there is surprising coherence in all your pictures and i sometimes share the same feeling altough technically every run should start from clean parameters... One question : do you keep the same init prompt (or very similar prompt) at each run ? Same init picture ? If someone dare we should ask the RiverHaveWings on her discord if this hypothesis make sense or not. Perhaps in each session there are some hidden parameter in static states until you completely refresh / restart the notebook...
I did run the images in the same session and used the same prompt and seed. But what I observed here might be a fluke or coincidental. I did run other trials of the same prompt, but I wasn't able to replicate it.
Yeah, that what I was expecting in my runs. Since the seed where not set to have same starting point at each time, I was expecting different output. Maybe, something happen when I am running the model that causes it to produce my images.
Maybe, there is something in the model parameters that collates what we are running until the notebook refresh. I might try it run and try to replicate.
Uhm, when I started to mess with this AI, expected a fresh start in every run, clean parameters and so... but it always drops similar results for a same prompt, and for different prompts and seeds, it still keeping same style along all the session. That's how I manage to create series in a same style. So I still thinking that it learns from the prompts and own results in a session. Maybe it keeps data in memory, until one turns off the AI, and Colab shuts down the VM... I don't know
And yeah guys, would be awesome to ask to the creator of this notebook... but I feel like an ancient peasant daring to go to bother some kind of deity lol
By experience i would say that the prompt is the main driver of the algorithm attention. If you don't modify it, each run will be 'similar' but not the same.
Seed is made to reproduce the exact same result. ( -1 = no seed) but if you put 123, and you re-run, you will get the same picture.
Modifying your prompt gives different results depending how heavy are those modification from previous run.
But in different sessions the same prompt and seed delivers different stuff, it is easy to notice using artists. You see what the AI is picking from them in each session. Whitin a session 99% of the runs picks the same stuff of each one. But in different sessions, at least, in my short time experience, it picks different stuff, even with same prompt and seed.
In my notebooks seed -1 = random seed. It came like this in the original one picked up. I implemented the same for the MSE one, so if you eventually use them, consider they got random seed feature.
Dear @juecoree, I first heard of the genre called AI Art.
If you input information, is the act of AI creating a future dystopia in virtual reality an AI Art?
How will the future world created by AI in virtual reality affect us?
Is there any reason why we must believe in what AI predicts in the future?
I agree with you!
The future world created by AI will give us artistic impressions and complex emotions such as thrill, fear, and happiness.
At the same time, it will become a means of creating and determining values in the fields of politics, economy, and science.
I think AI art has two sides of science and art!
Your idea is interesting and great!
Yes, it can create a dystopian vision based on the prompt (text or statement).
Like most forecasting, AI can predict a future state of an object from a historical data, but It may not be what will actually happen.
Yes, it can give us those emotion. AI often produce are that sometimes awesome, disappointing, and even weird. Indeed, AI can be sued for economic and political policies as well.
Yes, it blur out the lines between science and art. Enjoy a slice of !PIZZA
Love the effects after the iterations. Certainly does look like a post apocalyptic scene.
Thanks for sharing.
Glad to hear that. Thanks you!