Just before the holidays I posted an experiment video built with genAI tools to explain the evolution of the metaverse that may not always be obvious, as people often wait for a big bang of a product. Instead we see the ever increasing digital transition of our real time interactions, just as we have done with maps to GPS location to full GPS with traffic and route finding. Which is itself s digital twin of the world that many of us interact with daily in our cars and on our phones. That video is in this post.
Following on from that is this one that shows further things that can be done with the very same assets that made a sequential video, but now split into a presentation layout using virtual space. Plus a little bit about how dynamic virtual worlds can be. The primary message is that a GenAi video in a virtual world is not that far from being an entire virtual world to explore itself. We explore 3d data all the time ourselves in games, and in mapping too. Sometimes we do need to be spoon fed content, as in a video or a ppt deck, but other times it’s better to look and experience at your own pace. This virtual world concept for presentations is not a new one. Back in 2009 I wrote about trying a different presentation style laying out panels in Second Life, and also used it for rehearsing my first of many TV slots too in 2010 this one on 3D printers. Now what has become easier is the creation of the content in the first place. Well, I say easier, it’s different, still lots of trial and error and you need a bit of a vision for what you are trying to do.
I used my spatial.io account for this, but also to extend tech experiment I used a custom deployment through Unity to see how it all worked from that point of view too. So this is a mix of native spatial.io tools and their base world items, and some extras that I pumped int through Unity. (They have recently changed their licensing and access to not posting the spatial link as it may need some work 🙂 )
Reusing metaverse content in the metaverse to describe the metaverse….
Achievement unlocked. I just ran a local only version of a #genAI LLM and gave it the text of my 2 sci-fi novels Reconfigure and Cont3xt. Amongst other things I have had a conversation with my lead character Roisin!. Separately (as in the photo below) I also asked if the books helped solve if we live in a simulation. A great muse to chat about the potential of the 3rd book.reconfigurebook.co.uk Not perfect, but I can let it know what it’s not got quite right and also helps me remember the intense process of writing them in 2015, as they flowed onto the page like binge watching a boxed set.
I used the https://www.nomic.ai/gpt4all and just added one of Llama models to it. Giving it a fresh direct with a copy of the book PDFs was enough to get going. This is a MBP M2 chip machine, but there was no delay in having a conversation or diving right into the text or the personality of Roisin. Words are obviously much quicker to process than generating images or video.
AI Roisin picked up on some of her mannerisms in the books and played heavily on the various situations she has encountered. A lot of the book is about her inner voice and intentions so genAI had a lot to go on.
The wider world of the books, the tech and the philosophical elements of the story is something that it was not always getting quite right. Things changed a bit when it said it was trying not to generate spoilers, and I pointed out I wrote it, so the LLM changed tone and intention a little. It was the usual thing of asking it to describe something is it tells you x and y but not z. You mention z and it’s “sorry my mistake yes you are right”. However for a scoot through the lore, the background, some of the other characters this is all good. People may have read the book and got a different feel for something so it’s good to not treat it too rigidly.
My favourite part was when Roisin switched to whispering some extra details about something, an out loud statement followed by a psst…. listen type of moment. I have only had a few tech powered moments of that impact ever.
As with my previous renders of Roisin as an image and a video I am looking forward the ongoing evolution of this so I can hang out in a metaverse version of my created world and characters.
I recently gave a BCS presentation online for the Animation and Games specialist group and anyone else who wanted to come along where I kind of took some of the individual subjects that I have been engaged with in emerging technology and tried to describe them all in context with one another. It was a bit of a mad thing to try and do, but I also looked at how we might all be able to understand some of the technology advances by cutting through the jargon. It got a moderately philosophical with both fractal thinking and bringing in ying/yang concepts that my brain is pondering more due to learning Tai Chi. Quite a combination?
The presentation is now available on YouTube if you want to take a dive into “IoT, 5g, AI/GenAI, Cloud streaming, Edge computing, metaverse, Spatial computing, AR, VR, XR, quantum computing, industry 4.0/5.0, Brain computer interfaces (BCI), CRISPR, open source, crypto, Web 3.0 (the list goes on)”
The full BCS page with all the blurb and a download of the deck (minus the video element) is here
Warning, also has adverts for my books Reconfigure and Cont3xt, also related to all these concepts 🙂
As an industry analyst at a well known company (i.e. this is not about feeding edge as it is just a place to blog and hold the rights for my novels now) covering all things Metaverse, industrial, enterprise and consumer, I know there are few newer companies in the space that may not know what being an analyst means to them, so here is an updated and repurposed blog post I wrote a few years ago else where about what an industry analyst does and why you might want to talk to us. (about any of our coverage areas not just metaverse). Here is the who, what, where and when raison d’ĂŞtre for industry analysts as I see it.
Midjourney Cyberpunk Me
Who?
I been an industry analyst for nearly 8 years, but I have been in the emerging technology industry for well over 30 years as a software engineer and architect and as a writer and presenter. It is not uncommon for an industry analyst to have a lot of field experience, I used to brief analysts during my corporate life as IBM too. Equally many analysts are from a journalistic or statistics background, trained in finding and sharing facts and figures. Of course, there are many personality types within the profession overall, though in general the analyst profession is a people based one developing contacts and relationships across industry areas. Analysts have to take on board a lot of information but cut through the marketing hype to find patterns and facts based on their experience. Not all analysts are going to be long in the tooth like me, but it often helps. In the case of this metaverse wave having been what might now be called an influencer of the 2006 enterprise virtual worlds wave as a metaverse evangelist, putting Wimbledon into second life and developing solutions in the pre IoT connectivity days of things with the then new MQTT, not to mention early days of many current technology trends of web, e-commerce, social media, blogging and personally in gaming. My research agenda primarily covers emerging technology, where everything old is new again. An important thing to add, and certainly one that is true of my group is that analysts maintain integrity and impartiality by being separated from the commercial side of the company, though we do do commercial work, we are not do pay for play where I work. The content of what we cover and say is not based on how much clients pay, nor is it is based on how much the analyst relations department spends on a dinner. Everyone has their own motivation and connection to a subject but, impartiality, trust in treating off the record discussion and, building a reputation is core for our analysts.
What?
Large enterprises and the smallest of start-ups, and everyone in between, share a need to understand the market they are currently in, or look to move to. What is the competition up to? What is resonating in a market? What is going to totally disrupt the current plan? Start-ups need attention and connections to raise funding, enterprises need ongoing growth for shareholders and investors. Analyst companies are there to help provide answers and perspective on these sorts of questions. A large corporate entity may well have a competitive analysis department, but they will be focussed on the other big companies, less the quirky start-up set to make a dent in the industry. Smaller companies are busy just trying to do what they do, and may not have the time to look up and see what is going on around them. Analysts are always across a spectrum of companies, sizes and industry types. Whether it is writing regular short form reports on companies and their latest services, longer form reports across an entire industry sector, running ongoing surveys across thousands of industry types for directional data, custom consulting work, webinars, presentations, offering Merger and Acquisition (M&A) due diligence or just a quick call to answer a client question, analysts offer their considered opinions, backed by experience and data.
One of the other things that is important to consider, especially as a start-up, is to be in the minds of relevant analysts. We talk to people all the time, and suddenly a subject might come up, it may be a complete tangent, which is where we pattern match to say “I saw this really interesting approach from…”. Those conversations might be with VC’s, with companies looking for partners or with potential customers of the company.
Where?
The actual answer to this is anywhere. We take briefings from companies, typically for around 30mins – 1 hour on the phone, over web video conference, in person at trade shows. We try to ensure we do talk to companies, not just look at the web or a press release, as I mentioned above this is a people business. Trade shows such as the huge MWC bread and butter to analysts and for me Hannover Messe and Augmented World Expo are also important. By Example my experience of MWC in 2019, was 30-minute meetings scheduled with 30 mins walking time between them for 3 days, I logged 10 miles a day walking just in the conference centre. That is a lot of conversations and presentations from a varied set of companies.Â
Social media has always been a useful place for me personally and professionally since the early days, and analysts are often to be found there now. I am always willing to hear about interesting things on twitter/X as @epredator, on linkedin https://www.linkedin.com/in/epredator/ (and now mastodon and many other places).
When?
Anytime you need to know something across your industry, or you need the industry to know about something you are doing or about to do that’s when you can benefit from an analyst company, or multiple analysts’ perspective. A start-up may be in stealth, not ready to announce yet, and that is where analyst’s integrity is key, tell us what you are doing off the record and that may well lead to some suggestions that help or when you come out of stealth a better description from us as we share what you are doing.
Whilst we are separated from the commercial side of things, we are aware that what we produce is the core product for the company. Companies pay to access the basic reports, long form reports and detailed survey data is also charged for at a different rate, as is our time on the phone answering questions or doing custom consulting. However, companies telling us things is not something that anyone pays to do, and, in those conversations, we do often share our thoughts, so you can get something from a briefing, no need to just broadcast what you do. A final tip, if you are publicly known, have as much information as possible easily accessible, in a slide deck, or on a website. We all take notes during conversations but being able to look things up after the event is important. Who is the company, where were they founded, not just how cool the product is has great importance, this is a people business.
This month was the AGM for the BCS Animation and Games specialist group that I have been chairing for a very long while now. I gave a presentation from a personal view point (this is not a work presentation and I make that clear in the disclaimers, though it is what I work in too of course), on the advances in Metaverse and GenAI content creation. The full YouTube version is below but the link to the blurb and bio (and the video) at the BCS is here
We are always looking for presenters to come and share some ideas with our specialist group around all things games, animation, metaverse, esports etc, so if you are interested ping me there is a slot waiting for you. We sometimes get a big crowd, other times smaller ones but with the videos published like this it can be a useful thing to do and share.
For those of you who don’t know, BCS (formerly British Computer Society) Chartered Institute for IT is a UK based (but worldwide membership) professional body for anyone in the tech industry. It exists at all levels from just getting going in the business to Fellows with vast amounts of experience and willingness to help. It was part of my professional certification whilst at IBM and I then also became a certifier whilst there too. Volunteering and sharing ideas, such as this presentation, is one of the many ways to get involved (you don’t have to do this). It benefits you as an individual but also elevates tech roles within enterprises and organizations you work in.
Firstly, yes it looks like its a while since I posted on here, but that’s the challenge when work life and social/hobby life are one and the same thing. I have to save all the really good stuff for my professional output. However, I bought myself what is my 16th VR or AR headset with the Meta Quest 3 and as a long time user of such kit I wanted to share my personal (and only personal) thoughts about this bit of kit.
Quest 3 box
Headset and controllers
Looking like the Aliens from Toy Story 3 with 3 eyes wearing a Quest 3
Our Quest 2 has been kept in our kitchen ready for anyone in the family to use it since it arrived. The kitchen is the only place with just about enough free space to wave arms and make the odd step safely in VR. Though as I may have explained before there is a dent in the fridge from a side fist I did playing Superhot VR when I was training for my 2nd Degree Black Belt in Choi Kwang Do. The Quest 2 went straight to VR, obscuring the world (that was it’s job after all) but the guardian that we manually drew around the kitchen (often repeating that process) tended to keep us in bounds. Picking up the controllers and getting them on the right hands in the right way was always a bit fiddly, like plugging in a USB A and always getting it wrong the first two times despite there only being two ways to do it.
The Quest 3 on the other hand, starts with colour pass through, you put it on (over glasses too as they fit in the headset), and just see a slightly blurry version of where you actually are, its much quicker to get in and stood in the right place. (Quest 2 has pass through but it’s very disorientating and B&W). The new hand controllers are much easier to see and to pick up the right way as they don’t have the massive halo loops of plastic around them for sensors. The headset also does the guardian and scanning around for things on its own, you don’t have to (though you can) draw a line around yourself, which in the Quest 2 seemed like some sort of weird religious or mystical incantation involving a border of salt.
The Quest 3 is also light and smaller than the 2, or at least feels like it, I haven’t weighed them. This makes the whole experience to just get going much less faff an bother. It is a similar leap to when we had to place external sensors and try and line everything up before playing anything.
Going into a VR game of applications is pretty much the same across headsets, though it is noticeably crisper and faster/flashier as all new kit tends to be. Though now games win particular are being upgraded to have richer visuals, which in turn will start slowing it all down again. I plumped for the lower spec memory of 128Gb as mostly the family plays Beat Sabre and Pistol Whip, but I now have some games that seems to be 13Gb so it will eat it up more quickly now.
The Mixed Reality (MR) elements of the Quest 3 blending the pass through view of the world with digital elements anchored in that view is the clever bit. This is augmented reality but calling it mixed reality is as accurate. The view you see is virtual, its a camera showing you the physical world, it is able to scan the environment and build a digital mesh of it knowing what are walls, surfaces and objects in the view. It does in the example first steps app asking you to look around you and you see it overlay what it considers the mesh of the world. It’s very matrix, and deliberately visual, not really any need to show the user other than to intrigue them. The demo app then causes a break in the ceiling with bits falling on the floor and giving a view of a space environment and another planet in the distance. Through this hole drops a spaceship that lands (in my case on the the kitchen floor). Being in pass through makes it easy to walk around not bumping into physical things, as you can see them, so get a better look up into the hole in the ceiling. Over the course of the demo tribble like fluffy things break through the walls and run around you space, they drop behind things like the breakfast bar in our kitchen and you can’t see them unless you go and look over. They creatures also run along the walls and other surfaces. It really is quite effective as a cartoon comedy demo of full AR. as this is not holding up a tablet or phone the field of view and turning of you own head and hence view is 100% natural.
The wonderful old painting application Vermillion had some upgrades, so now in full MR the easel and paint brushes can be in your own room and now even better you can hang you paintings on you physical wall and they will stay there if you want them too. You can see family walking into the room and talk to them, though it’s a little odd for them to know whether you can see them or not, having been used to giving the VR headset a wide birth (as the fridge didn’t above) 🙂
This Quest 3 for the kitchen liberates by Quest 2 for my gaming PC, which in turn retires the tethered Rift/S. It also means we can play multiplayer games with two of us in the house. The main account can share their apps (though only on one headset). I shared the Quest 3 apps and unshared on Quest 2 but we were able (with me logged into Quest 2 and Predlet 2.0 using his account on Quest 3) to play a good few rounds of the VR Walkabout Mini-golf and some multiplayer beat Sabre. I say a good few rounds but the battery life is pretty short on both headsets. This can be sorted by attaching a very long USB cable to the power outlet, but that sort of removes the untethered nature of the headset.
This Quest 3 and pass through is obviously a much cheaper version of what the Apple Vision Pro is aiming to do, but it’s out first, it works and it has a library of interesting things to engage with. Though the really new stuff like Assassin’s Creed and Asgards Wrath II (Remake) are not out at release. So it’s more to play VR things and the odd MR update at the moment. I say out first but pass through has been the mission of commercially available Varjo headsets for a while.
One other thing pass through allows for is being able to see you phone (sort of as its a bit warped). This is very useful as trying to buy things in the store in VR doesn’t work as it needs the phone app to do all the verification. This used to mean taking the headset off, now it means double le tapping the headset for pass through and finding your phone. That’s a silly use case as it should just work in the environment or represent the phone and its OS in the environment, but that’s payment rails for you.
In short, it’s good, worth it and I like it. IO am looking forward to more proper AR/MR apps and experiences, whilst enjoying the brand new ones like Samba De Amigo that just launched (again not new as we played that a lot on the Sega Dreamcast with a set of maraca controllers way back).. and pose….. Happy memories reborn 🙂
Having my books Reconfigure and Cont3xt I am always using them and the story I created as a way to explore other mediums. I started drawing a few years ago to see if I could maybe go manga with it, that included buying some action figure mini drawing mannequins that have some tech and weapon props. I created some meta humans and have a unity based mocap suit from a kickstarter to see if I could produce anime. Audio voice generation has been another thing I tried. Each of these projects would be a significant amount of time, effort and probably cost too, but delving in as a mini proof of concept gives me an appreciation of what would be needed.
Midjourney and the AI image generators have offered another option to explore, an very quickly too. One of the challenges is to get the same character to appear in each generated image (i.e. story board). I decided to sort that out later in this POC. Though I did suggest that the actress to play Roisin should be Ruby Rose to get a potential consistent likeness.
Roisin at the start of Reconfigure in a bit of a last minute situation getting off the grid
Having got this image, which was pretty good for a first attempt, with very little text in the prompt I popped over to runway.ml and tried its new gen-2 image to video generation. This time with NO PROMPT at at all! It decided she should look up and stare into the distance, which in the book she is doing as she is being chased and very aware of her surroundings as she tries to outwit those after the amazing tech she has discovered. They only generate short clips on the free accounts, but those few seconds I found to be quite astounding and bewitching.
GenAI is obviously on a major drive into all sorts of areas, but the ability to potentially use tools like this to create a film version of my books on my own time and budget is emerging as a strong contender, unless Netflix or Prime decide they would like to do it instead 🙂
In 2016 I wrote Cont3xt the follow up to Reconfigure and part of that was an upgrade our lead character gets. She has access to an API that can manipulate the physical world like the virtual, she builds that up into a Unity application for her smart phone in the first book. The second she gets an EyeBlend headset, state of the art (and still just a little ahead of Apple and co’s attempts. So she ports the Unity app and upgrades the messaging system for this high bandwidth device and to enhance the experience. Rather like all the developers are doing right at the moment. Life imitates art 🙂 Read Reconfigure first, but this might be a topical time to check out Cont3xt whilst you wait for the Apple demo in a local store next year. (Just ÂŁ0.99 each for these adventures 🙂 )
“An EyeBlend? They are not on general release yet? How did you get that?” Her eyes lit up wildly as Roisin took in the news.Â
“Oh you know, I have a few friends who hack for a living.” Alex replied with an air of comedy superiority.
“You sure do! The EyeBlend is an awesome piece of kit. It does full blended reality applications. It uses field effect projection with live object tracking!” She said enthusiastically. “They call it a hologram, though it really isn’t as such. You can forgive that though as the effect is the same, apparently.” Roisin was on her tech turf again.
I had the honour of hosting my good friends from Gamesatwork.biz for a BCS animation and games event to hear about how they have managed to create compelling content in a weekly podcast for what is not approaching 400 episodes! The official BCS event link page and links to everything else about the BCS is here but I have added the youtube directly. Its very free form, not scripted and was a entertaining to be the interviewer, especially as I know Michael, Michael and Andy just know what to say and when to say it.
BCS Animation and Games hosts Games at work dot biz team for a free form chat about the show
As the guys explain a lot of the content is based on us all sending them things we are spotting or are busy doing and enjoying. All the podcasts episodes have their own show notes and links so after watching this go check those out and see the sort of thing they do, or vice versa and then start sharing stuff with them, the more the merrier. Also if you are in tech, check out the BCS as a professional org.
It always amazes me how there is always something to learn or be surprised about tech, you think you have a handle on it and then you see or somehow sense something else. I recently got hit with a thought about the relative sizes of an avatar in a shared virtual space/metaverse application and what that can be used for. I have often talked about the scale of virtual worlds being a mutable thing, we can be sub atomic or at the scale of a universe in a virtual environment. I have also talked about being the room or place, i.e. a sort of dungeon master approach controlling the room (as a version of an avatar) whilst others experience something about that place.
Avatars of all sizes
The first experience that got me thinking about relative avatar size was getting to play Beat Saber as a 3 player VR game. I had done 2 player, here you see your fellow player off to the side facing you but experiencing the same song and blocks. You are so busy playing and concentrating, but it is a good experience to be shared. However once over 2 people, at 3 in this case, you are all on your own track but facing a common central point, like being on a star. The player that is doing best has their avatar zoomed and projected up towards that central point as a reminder you need to work harder. Its a great mechanic and is then using the avatar size as a communication mechanism, i.e. they are better at Beat Saber than you.
The next experience was to properly play the VR dungeon crawling turn based dice and card Demeo as a multiplayer. This game has you and your friends gathered around a board decked with figurines. You are represented by you hands, to pick up the pieces and moves them, or to roll the dice, and by a set of glasses or small mask. The avatar is not a whole thing, no need for legs etc. The board is the thing you all want to see. Each player can change not only the direction of the board just for them, by dragging it around but also zoom in and out to see the lay of the lay or get right in and look at how the characters have been digitally painted. The game is collaborative and turn based, and you get to see the other players had and mask avatars. Here though is the twist, you also get a sense of whether the other person is stood looking around the table top or if they have zoomed close because the avatar you see of them scales up and down in size according to their view of the world. Not only can you see the direction they are looking but how detailed their view might be. If you are zoomed in and you look up you see the giant avatar of your fellow player looking at you. This is all very fluid, gameplay is not messed up by it, and it shows a level of body language that only really exists in metaverse style applications. The VR element makes it feel even better but the effect works on 2d screens too.
We often talk about sense of presence, remember you were next to someone in a virtual place, but this is another dynamic that is obvious, but only obvious now having experienced it. Anyway, that’s the thing I learned earlier this week. Also in order to find a picture to express the image above started as one DALL-E sentence for the AI generation, but I also tinkered around with the new web editing that lets you add new frames of generation to existing images, creating AI generated composites!. The stately home was an accident, but somehow looks very much like my old 2006 metaverse evangelist physical and virtual home of IBM Hursley. The ability to create an image and a feeling for this post, not just grab a screenshot for the games (which is always tricky to get what you want anyway when it involves other players), is also rather cool and is the first time I have created an image for a subject, rather than the subject be image generation. The future is arriving fast!
Anyway, avatars, not always what you think they are are they? 🙂
