Just before the holidays I posted an experiment video built with genAI tools to explain the evolution of the metaverse that may not always be obvious, as people often wait for a big bang of a product. Instead we see the ever increasing digital transition of our real time interactions, just as we have done with maps to GPS location to full GPS with traffic and route finding. Which is itself s digital twin of the world that many of us interact with daily in our cars and on our phones. That video is in this post.
Following on from that is this one that shows further things that can be done with the very same assets that made a sequential video, but now split into a presentation layout using virtual space. Plus a little bit about how dynamic virtual worlds can be. The primary message is that a GenAi video in a virtual world is not that far from being an entire virtual world to explore itself. We explore 3d data all the time ourselves in games, and in mapping too. Sometimes we do need to be spoon fed content, as in a video or a ppt deck, but other times it’s better to look and experience at your own pace. This virtual world concept for presentations is not a new one. Back in 2009 I wrote about trying a different presentation style laying out panels in Second Life, and also used it for rehearsing my first of many TV slots too in 2010 this one on 3D printers. Now what has become easier is the creation of the content in the first place. Well, I say easier, it’s different, still lots of trial and error and you need a bit of a vision for what you are trying to do.
I used my spatial.io account for this, but also to extend tech experiment I used a custom deployment through Unity to see how it all worked from that point of view too. So this is a mix of native spatial.io tools and their base world items, and some extras that I pumped int through Unity. (They have recently changed their licensing and access to not posting the spatial link as it may need some work 🙂 )
Reusing metaverse content in the metaverse to describe the metaverse….
Achievement unlocked. I just ran a local only version of a #genAI LLM and gave it the text of my 2 sci-fi novels Reconfigure and Cont3xt. Amongst other things I have had a conversation with my lead character Roisin!. Separately (as in the photo below) I also asked if the books helped solve if we live in a simulation. A great muse to chat about the potential of the 3rd book.reconfigurebook.co.uk Not perfect, but I can let it know what it’s not got quite right and also helps me remember the intense process of writing them in 2015, as they flowed onto the page like binge watching a boxed set.
I used the https://www.nomic.ai/gpt4all and just added one of Llama models to it. Giving it a fresh direct with a copy of the book PDFs was enough to get going. This is a MBP M2 chip machine, but there was no delay in having a conversation or diving right into the text or the personality of Roisin. Words are obviously much quicker to process than generating images or video.
AI Roisin picked up on some of her mannerisms in the books and played heavily on the various situations she has encountered. A lot of the book is about her inner voice and intentions so genAI had a lot to go on.
The wider world of the books, the tech and the philosophical elements of the story is something that it was not always getting quite right. Things changed a bit when it said it was trying not to generate spoilers, and I pointed out I wrote it, so the LLM changed tone and intention a little. It was the usual thing of asking it to describe something is it tells you x and y but not z. You mention z and it’s “sorry my mistake yes you are right”. However for a scoot through the lore, the background, some of the other characters this is all good. People may have read the book and got a different feel for something so it’s good to not treat it too rigidly.
My favourite part was when Roisin switched to whispering some extra details about something, an out loud statement followed by a psst…. listen type of moment. I have only had a few tech powered moments of that impact ever.
As with my previous renders of Roisin as an image and a video I am looking forward the ongoing evolution of this so I can hang out in a metaverse version of my created world and characters.
I recently gave a BCS presentation online for the Animation and Games specialist group and anyone else who wanted to come along where I kind of took some of the individual subjects that I have been engaged with in emerging technology and tried to describe them all in context with one another. It was a bit of a mad thing to try and do, but I also looked at how we might all be able to understand some of the technology advances by cutting through the jargon. It got a moderately philosophical with both fractal thinking and bringing in ying/yang concepts that my brain is pondering more due to learning Tai Chi. Quite a combination?
The presentation is now available on YouTube if you want to take a dive into “IoT, 5g, AI/GenAI, Cloud streaming, Edge computing, metaverse, Spatial computing, AR, VR, XR, quantum computing, industry 4.0/5.0, Brain computer interfaces (BCI), CRISPR, open source, crypto, Web 3.0 (the list goes on)”
The full BCS page with all the blurb and a download of the deck (minus the video element) is here
Warning, also has adverts for my books Reconfigure and Cont3xt, also related to all these concepts 🙂
This month was the AGM for the BCS Animation and Games specialist group that I have been chairing for a very long while now. I gave a presentation from a personal view point (this is not a work presentation and I make that clear in the disclaimers, though it is what I work in too of course), on the advances in Metaverse and GenAI content creation. The full YouTube version is below but the link to the blurb and bio (and the video) at the BCS is here
We are always looking for presenters to come and share some ideas with our specialist group around all things games, animation, metaverse, esports etc, so if you are interested ping me there is a slot waiting for you. We sometimes get a big crowd, other times smaller ones but with the videos published like this it can be a useful thing to do and share.
For those of you who don’t know, BCS (formerly British Computer Society) Chartered Institute for IT is a UK based (but worldwide membership) professional body for anyone in the tech industry. It exists at all levels from just getting going in the business to Fellows with vast amounts of experience and willingness to help. It was part of my professional certification whilst at IBM and I then also became a certifier whilst there too. Volunteering and sharing ideas, such as this presentation, is one of the many ways to get involved (you don’t have to do this). It benefits you as an individual but also elevates tech roles within enterprises and organizations you work in.
Having my books Reconfigure and Cont3xt I am always using them and the story I created as a way to explore other mediums. I started drawing a few years ago to see if I could maybe go manga with it, that included buying some action figure mini drawing mannequins that have some tech and weapon props. I created some meta humans and have a unity based mocap suit from a kickstarter to see if I could produce anime. Audio voice generation has been another thing I tried. Each of these projects would be a significant amount of time, effort and probably cost too, but delving in as a mini proof of concept gives me an appreciation of what would be needed.
Midjourney and the AI image generators have offered another option to explore, an very quickly too. One of the challenges is to get the same character to appear in each generated image (i.e. story board). I decided to sort that out later in this POC. Though I did suggest that the actress to play Roisin should be Ruby Rose to get a potential consistent likeness.
Roisin at the start of Reconfigure in a bit of a last minute situation getting off the grid
Having got this image, which was pretty good for a first attempt, with very little text in the prompt I popped over to runway.ml and tried its new gen-2 image to video generation. This time with NO PROMPT at at all! It decided she should look up and stare into the distance, which in the book she is doing as she is being chased and very aware of her surroundings as she tries to outwit those after the amazing tech she has discovered. They only generate short clips on the free accounts, but those few seconds I found to be quite astounding and bewitching.
GenAI is obviously on a major drive into all sorts of areas, but the ability to potentially use tools like this to create a film version of my books on my own time and budget is emerging as a strong contender, unless Netflix or Prime decide they would like to do it instead 🙂
In 2016 I wrote Cont3xt the follow up to Reconfigure and part of that was an upgrade our lead character gets. She has access to an API that can manipulate the physical world like the virtual, she builds that up into a Unity application for her smart phone in the first book. The second she gets an EyeBlend headset, state of the art (and still just a little ahead of Apple and co’s attempts. So she ports the Unity app and upgrades the messaging system for this high bandwidth device and to enhance the experience. Rather like all the developers are doing right at the moment. Life imitates art 🙂 Read Reconfigure first, but this might be a topical time to check out Cont3xt whilst you wait for the Apple demo in a local store next year. (Just £0.99 each for these adventures 🙂 )
“An EyeBlend? They are not on general release yet? How did you get that?” Her eyes lit up wildly as Roisin took in the news.
“Oh you know, I have a few friends who hack for a living.” Alex replied with an air of comedy superiority.
“You sure do! The EyeBlend is an awesome piece of kit. It does full blended reality applications. It uses field effect projection with live object tracking!” She said enthusiastically. “They call it a hologram, though it really isn’t as such. You can forgive that though as the effect is the same, apparently.” Roisin was on her tech turf again.
My social feeds were getting full of people trying out a version of the GPT3 AI for text by OpenAI on https://chat.openai.com/chat so I thought I would have a look at it. It is as amazing as the AI art generation, not a sentient being but a tool that when asked the write questions gives interesting answers.
Whilst essay style responses are interesting I asked it a rather odd question with a style slant towards it and this is what I got. Not hilariously funny, but just consider that the characters are as you would expect them to be. There is a sense of trying to get out of doing something, then getting caught out and sent back to the beginning.
“Write a Blackadder sketch set in a 20th century office environment”
Blackadder: (sitting at his desk, staring at a stack of papers) Baldrick, I’m drowning in paperwork.
Baldrick: (leaning against the doorframe) Yes, sir.
Blackadder: (sighing) It seems like no matter how much I do, there’s always more to do. And on top of that, the boss is breathing down my neck.
Baldrick: (nodding) Yes, sir.
Blackadder: (slamming his fist on the desk) I need a break, Baldrick. I need to get out of this office and do something fun for a change.
Baldrick: (smirking) I’ve got just the thing, sir. How about we play a game of darts?
Blackadder: (raising an eyebrow) Darts? In the office? Are you mad, Baldrick?
Baldrick: (shrugging) Why not, sir? It’ll be a nice change of pace, and it’ll take your mind off all this paperwork.
Blackadder: (considering) Hmm, I suppose you have a point. But we’ll have to be careful not to break anything.
Baldrick: (grinning) Don’t worry, sir. I’ll go get the darts.
(Baldrick exits the room and returns a moment later with a dartboard and a set of darts)
Blackadder: (picking up a dart) Alright, Baldrick. Let’s see who the better marksman is.
(They start playing darts, with Blackadder proving to be the better player)
Blackadder: (throwing a dart and hitting the bullseye) Ha! Another bullseye, Baldrick. You’ll have to try harder than that if you want to beat me.
Baldrick: (throwing a dart and missing the board entirely) Oops, sorry sir.
Blackadder: (rolling his eyes) Never mind, Baldrick. At least we’re having fun.
(Just then, the boss enters the room)
Boss: Blackadder! What is going on in here?
Blackadder: (turning to face the boss) Ah, just taking a break from all the paperwork, sir. Playing a friendly game of darts.
Boss: (scowling) Darts? In the office? This is highly inappropriate, Blackadder. I expect better from you.
Blackadder: (apologetically) My apologies, sir. It won’t happen again.
Boss: (sternly) See that it doesn’t. Now, get back to work, both of you.
(Blackadder and Baldrick sheepishly return to their desks and resume their work, with Blackadder grumbling about the unfairness of it all)
It’s not just long form that this bot can produce I also asked it for something shorter and to the point
“Write a haiku about cars”
Cars speed down the road,
Metal beasts with roaring engines,
Nature takes a backseat.
Next I chose to mix it with some AI image generation (this time using photo leap on the iPhone)
Create a play with two people discussing the Metaverse, one person is positive the other person is negative
Act I:
Person 1: Have you heard about the Metaverse? It’s this amazing virtual world where anything is possible.
Person 2: Yeah, I’ve heard of it. But it’s just a bunch of hype. It’s never going to happen.
Person 1: Are you kidding me? The technology is already here. It’s just a matter of time before we can all access it.
Person 2: I don’t know. It sounds like a utopia, but I think it’s more likely to become a dystopian nightmare.
Person 1: That’s just fear-mongering. The Metaverse has the potential to revolutionize the way we live and interact with each other.
Act II:
Person 2: But what about the consequences? What happens when people start spending all their time in the virtual world instead of the real one?
Person 1: That’s a valid concern, but I think it’s a risk worth taking. The potential benefits are just too great to ignore.
Person 2: But what about the loss of privacy? In the Metaverse, everything about us will be out in the open for anyone to see.
Person 1: That’s true, but I think the benefits will outweigh the drawbacks. The Metaverse will allow us to connect with people in ways we never thought possible.
Act III:
Person 2: But what about the cost? Creating and maintaining the Metaverse is going to be incredibly expensive.
Person 1: That’s a short-sighted view. The Metaverse has the potential to generate huge amounts of wealth and create countless new jobs.
Person 2: But what about the impact on the environment? The amount of energy needed to power the Metaverse will be enormous.
Person 1: Again, I think the benefits will outweigh the drawbacks. The Metaverse will allow us to explore new frontiers and push the boundaries of what’s possible.
Act IV:
Person 2: I still have my doubts. The Metaverse sounds like a pipe dream to me.
Person 1: That’s okay. Not everyone has to believe in it. But I have faith that the Metaverse is the future, and I’m excited to see where it takes us.
Person 2: I hope you’re right. But I’m still skeptical.
Person 1: That’s okay. Time will tell. In the meantime, I’ll continue to dream of the possibilities the Metaverse holds.
Yet more wonders I did behold when I asked…
“Describe the Metaverse in the style of William Shakespeare”
In the style of William Shakespeare, the Metaverse may be described as a boundless realm of virtual reality, where spirits roam free and digital avatars take on form and substance. It is a world beyond the flesh, where imagination is the only limit and all things are possible. Here, one may forge new identities and explore uncharted territories, unfettered by the constraints of the physical world. In this vast and wondrous domain, the boundaries of time and space are mere illusions, and the possibilities for adventure and discovery are endless. So come, brave travelers, and enter the Metaverse, where you may revel in the boundless wonders of a virtual world, and unleash the full power of your imagination.
Very clever, and this is accelerating very quickly. Certainly something to keep an eye on!
Firstly, yes its been a while since I posted here, I could have done with an AI clone of me carrying on sharing interesting things. I was out mainly due to an annoying post-ailment “thing” (medical term) hitting my facial nerves which meant for months any screen time trying to read or write was visually and mentally uncomfortable. That was followed with a dose of Covid that knocked me out for a while but also seems to have had a potential positive impact on whatever else was going on. Anyway, I am back. Anything happened the past few months, I mean 1/2 a year?
I suddenly felt compelled to blog about some of the marvellous work going on with AI image generation of late with things such as Dall-e and Midjourney becoming available to more people. I have so far only used Midjourney, which is accessed through Discord. In the relevant channel you simply type the command /imagine followed by any descriptive text you fancy and it will generate four images based on that, with often surprising results. Any of those images can be selected as the root to generate more in that style, or you can upscale, get a big version, of one you favourite.
Handy hint- The mobile version of Discord really likes to scroll to new items, midjourney works by updating your post to the channel. As so many posts are being generated every few seconds you might think you can leave the app on your post to see the results but very quickly you will zoom up the screen. Constant scrolling back to try and catch your images being generated is a bit of a pain. However if you log into midjourney.com with Discord ID it will present you with your collection and lots of links to get back to do things with them. The Discord search doesn’t seem to work on userid etc so it took me a while to figure this out.
After a few attempts, and with one eye on the political situation and crisis faced in the UK I asked mid journey “an artist forced to stop painting in order to work in a bank” and got this wonderful set of images
an artist forced to stop painting in order to work in a bank
I also asked it to generate a protest by the people
a crowd of down trodden people protesting and rising up against the billionaires and corrupt politicians
Rather than continue the doom and gloom it was time to go a bit more techie 🙂 I asked it to show me directing the metaverse from a control centre
epredator directing the Metaverse evolution from a hi-tech digital operations center
An obvious course of action was to explore my Reconfigure and Cont3xt novels and this is where I was very impressed by whatever is going on with this AI. I asked to see “a cyberpunk girl manipulating the physical world using an augmented reality digital twin” and it seems that Roisin (@Axelweight) is in there somewhere 🙂
a cyberpunk girl manipulating the physical world using an augmented reality digital twin
This was worth exploring and picking couple of the images to then seed generating more similar to that which generated these two
a cyberpunk girl manipulating the physical world using an augmented reality digital twin (generation 2 from bottom left of gen 1)
And this one
a cyberpunk girl manipulating the physical world using an augmented reality digital twin (generation 2 from top right of gen 1)
These were good but the first version of the top right image in generation 1 was the one I did a more detail version of on its own. It could be the cover of the third book couldn’t it?
Midjourney generated potential book cover for third in the reconfigure trilogy. Of course one fo these from ore geometric results are more in line with my current cover designs asking “A world of Fractal Iterations, Quantum Computing and strange side effects opened up. It appeared to offer a programming interface to everything around her.”
A world of Fractal Iterations, Quantum Computing and strange side effects opened up. It appeared to offer a programming interface to everything around her.
In case this seems a all a bit too serious, I did ask a slightly weirder question of the AI and got this.
two ducks with a top hat on carrying a salmon
For a happy image how about this /imagine a bright green planet full of life
a bright green planet full of life
But back to the surreal and unusual and predlet 1.0 and I share a joke we “invented” when she was quite young, we were passing the pub in Lock’s Heath having been to the supermarket, she looked at the pavement and saw a creature. She said “dad look there’s a worm coming out of the pub” to which I replied “and he is legless”, and here it is
a worm legless outside a pub
As this kind of AI generation evolves from the static image to the moving one and then onto virtual world and the metaverse, what a wonderfully weird set of worlds we will be able to dream/nightmare up 🙂
As many people who pop along here will know back in 2006 (and even before that) I have been somewhat passionate in explaining and sharing what can be done with things like game technology, virtual worlds etc to help with human communication, that includes entertainment (which is often regarded as not a serious business), office type business, education, industrial digital twins the list goes on. Metaverse is a handy word to bundle all this together and whilst not tightly defined I think many of us know what we mean. This month I wrote a piece for the BCS which is available to read here and they have nicely highlighted the phrase ““Travelling and actual teleportation aside, do you think that we should be able to find better ways to feel we are with people and to have shared experiences?” Whatever you think the Metaverse is evolving to, just consider if the way you are doing things now is the best it could possibly be given the tech available. Things evolve and move, not always replacing everything behind them but adding.
Having sad that, it is worth reminding everyone that the original work in 2006 in Second Life was mostly about pulling data from other sources into the environment, and doing things with it, this eventually led to building Wimbledon’s 2006 course with the ball trajectory from Hawkeye rendered there.
All about data from elsewhere integrating into SL in 2006
There is an ongoing discussions is about interoperability of Metaverse or virtual world platforms. These range from is it desirable to have objects from world x appear in world y and then often hit the barrier of platform x has a completely different rendering engine to platform y. The former is a style and even a business question, why does an environment of any type exist, real, virtual, augmented, projected, whatever. This then leads to the technical wars around content being “in” a development or build environment. Importing of meshes, textures, animation rigs etc is one way to look at Metaverse evolution but I think it is also worth considering the potential for things like remote rendering, if the unifying standard for content sharing was actually containers, of potentially any size or shape and level of detail, the anything from environment x could appear in environment y and even interact with it to vary degrees depending on permissions or needs. My Nvidia RTX ray traced shiny car could appear in the most basic of 3d environments as I am rendering the car my end, and the virtual world is processing a container and stream. This is after all what we do when we video chat, each of our cameras is doing some work and being hosted by a basic integration layer. Yes I know 3d and collisions and physics and and etc. all make it more complex but it is worth considering that other ways of working can exist to the build and compile we are used to from games. Also, from what I remember of Snowcrash avatars are rendered by the person’s own device, those with worse computing power have more basic avatars in world? Also, we already did some of this back n 06/07 it just takes a different way of thinking.
Talking to thinking, literally, I was invited to join the Games at Work dot Biz podcast again and I was more than happy to give a shout out to a book by Prof Richard Bartle on virtual worlds and our god like powers over them. Checkout the podcast and links are here and on any player you have, give it a rating to help the guys too please.
Well what an interesting day yesterday was as Facebook went all out on being a Metaverse company and renamed itself to Meta. It also marked a point that crosses my straight unbiased industry analyst day job with my constancy of being a metaverse evangelist, a title I had in 2006 and maintain. These comments are mine alone and not that of the company I work for. I have lots more to share and discuss on all things Metaverse, but that’s something that people have to pay for nowadays.
However, I will say that I was really pleased to see a slick presentation on all things Metaverse from a multi billion dollar company like Facebook. Last year the same happened to some degree with Microsoft and Mesh at Ignite. The conversations have moved by many people from, “nah we don’t want to interact like that ever” to “no one company should own all this”. The Metaverse the industry and society needs does, as was said in the Meta keynote, need to be able to let people connect with both the physical and multiple virtual realities for a multitude of reasons and with many available ways to be who we are. I have, for many years, talked about how we have had to adjust to the limitations of technology to allow our communication at distance. We still type on qwerty keyboards designed as a layout in part to stop physical typewriter hammers jamming by spacing out the commonly used letters in words to avoid that issue. Not that we need to get rid of this form of typing, but to hold it up as the only way to interact with technology and data seems a little bit odd doesn’t it?
Back a decade or so ago I created a slide for a presentation on virtual worlds and the metaverse that was aimed at showing how far we might be able to go and why. It was in part a presentation about the fact it is not all avatars and islands. Those work, but there are many more things that come together. There are many forms of input, including neural interfaces (see below at the left), there are combinations of virtual to virtual, virtual to real (AR) and as I said at the end virtual to real to real, which is virtual objects becoming physical again (3d printing etc). It is about the connecting of people, a wetware grid, but threaded by narrative and context. The Meta presentation often showed these as quick interactions, thrown a basket ball, looking at a design etc, but the there is of course a longer thread, a digital thread if you like. It’s all very fractal in its nature, the patterns repeat at any scale.
One thing to note though is that everything feeds back into the entire loop, the arrows are multi directional. You might 3d print a “device” or prop you found in a virtual world interaction, but that itself might be an input device for another experience. Imagine if you 3D printed a modelling clay tool you bought in a virtual world then made a clay sculpture that you took a photo of for Flickr, or even scanned the sculpture in and made it a virtual object as a one off that you sold on an NFT market place which then saw it used as a prop in a virtual world movie, and so on. I used to talk about printing off a penny whistle and joining in a virtual band at an irish pub in Second Life. I hope you get the gist.
One key thing though, and it is slightly annoying Facebook have nicked the word but as you see below the entire loop is the meta loop (c 2009).
Describing the Meta loop in 2009 – Its not all avatars and worlds/islands
I know many of us old guard or original gangster, however OG you want to be, might be annoyed at the mega corp going Meta on us and riding high on our hard work persuading the world this is a good idea and the way we are all going. However, it is the way we are going, so we get to say “I told you so”, rather than multi billionaires. Money clearly isn’t everything 🙂 We, and I include all my fellow Eightbar people and surrounding community from back in 2006, we were already riding on top of some Double OG work. It’s the nature of waves of societal acceptance and tech development. The hope is that business and experiences that are born of the Metaverse with thrive and bring entertainment, education, excitement and benefit to the world. By metaverse business I mean not the ones building the underlying tech platforms, but the ones thriving on top of them, but then going on to influence the wave after this just as Facebook and co are today. It’s certainly not going away, so my sons birth certificate from 14 years ago saying father’s occupation “Metaverse Evangelist” might turn out to be of use to him someday.
I mentioned above that the physical world is an important part of the metaverse and in the Facebook/Meta pitch there was an instrumented house where the mugs and teapot that were carried in real life updated their position in the digital model. Well, that around the other way is the premise of my Reconfigure and Cont3ct novels. They are very metaverse, IoT, Quantum AR and VR related, but our hero Roisin gets to manipulate the physical world by altering the digital world. It works for an adventure, but it also keeps the “scifi set this afternoon” concept alive and it is not yet all done and dusted 🙂 Go take a look it’s very reasonably priced and might spark a few ideas on the back of the future that Meta are pushing. More here or see the ads not the right.
