I thought it would be interesting to revisit some of the Augmented Reality tools that have Unity3d packages for them. The first I headed for was Vuforia, now owned by PTC. I am using the free version. It all worked straight from the install, though there are a lot of features to tinker with.
I wanted to use an image marker of my own, so naturally I used the book cover. There is a lot of AR and VR in the story so it all makes sense.
For the 3d models I used the current trial of Fuse/Mixamo now part of the Adobe. It lets you build a model and export it in a nice friendly Unity3d fix format. You would be suprised how these various formats for models and animations hide so much complexity and weird tech problems though. Eventually I found a couple of animations that fitted the mood. This is not a full action sequence from the book, but it could be. I have some more vignettes in mind to experiment with. I am just upgrading my windows machine so I can use the Kinect 2.0 for some of my own motion capture again. Finding the right animation, and then trying to edit it is really hard work. The addition of motion, especially in humans, hits a different part of the brain I think. It makes lots of things look not quite right.
So here is the video of augmented reality on a cover of a physical copy of an ebook novel telling the story with lots of augmented reality in it, if thats not too meta. Roisin rather sassily taps her for while the Commander lays down the law to her.
Reconfigure and Cont3xt are available right now to enjoy, and if at all possible, PLEASE write a review on Amazon, it makes a massive difference to the take up of the books.
metaverse
Lucky 7 years – Feeding Edge birthday
Wow. It is seven years since I started Feeding Edge Ltd. That is quite a long while isn’t it? The past year has been a more difficult one with less work in the pipeline for most of it. It has meant I have had to take stock and look to do other things, whilst the World catches up. It does seem strange given the dawn of the new wave of Virtual Reality, and Augmented Reality that I have not managed to find the right people to engage my expertise and background in both regular technology and virtual worlds. In part that was because I was focussing on one major contract, when that dried up suddenly there was no where to go. It is starting from zero again to build up and sell who I am and what I do.
My other startup work has always been ticking along under the covers, as we try and work the system to find the right person with the right vision to fund what we have in mind. It is a big a glorious project, but it all takes time. Lots of no, yes but and even a few lets do it, followed by oh hang can’t now other stuff has come up.
On the summer holiday, I had a good long think about whether to give this all up and try and find a regular position back in corporate life, I was hit with a flash of inspiration for the science fiction concept. It was so obvious that I just had to give it a go. That is not to say I would not accept a well paid job with slightly more structure to help pay my way. However, the books flowed out of me. It was an incredibly exciting end to the year. Learning how to write and structure Reconfigure, how to package and build the ebook and the print version. How to release and then try and promote it. I have learned so much doing it that helps me personally, helps my business and also will help in any consulting work I do in the future. I realised too that the products of both Reconfigure and Cont3xt are like a CV for me. They represent a state of the virtual world, virtual reality, augmented reality and Internet of Things industry, combined with the coding and use of game tech that comes directly from my experiences, extrapolated for the purpose of story telling.
Write what you know, and that appears to be the future, in this case the near future.
This year I have also been continuing my journey in Choi Kwang Do. This time with a black suit on as a head instructor. It has led me to give talks to schools on science and why it is important, with a backdrop of Choi Kwang Do as a hook for them. I am constantly trying to evolve as a teacher and a student. Once again the reflective nature of the art was woven into the second book Cont3xt. I did not brand any of the martial arts action in the book as Choi Kwang Do as that may mis-represent the art and I don’t want to do that, but it did influence the start of the story with its more reflective elements, later on a degree of poetic licence kicked in, but the feelings of performing the moves is very real.
I have continued my pursuit of the unusual throughout the year. The books as a product provide, rather like the Feeding Edge logo has in the past, a vehicle to explore ideas.
I still really like my Forza 6 book branded Lambo, demonstrating the concept of digital in world product placement.
If you have read the books, and if not why not? they are only 99p, you will know that Roisin like Marmite. Why not ? I like Marmite, again write what you know. It became a vehicle and an ongoing thread in the stories, and even a bit of a calling card. It is a real world brand, so that can be tricky, but I think I use it in a positive way, as well as showing that not everyone is a fan. So the it is just another real world hook to make the science fiction elements believable. So I was really pleased when i saw that Marmite had a print your own label customisation. It is print on demand Marmite, just as my books are print on demand. It uses the web and the internet to accept the order and the then there is physical delivery. I know its a bit meta but thats the same pattern Roisin uses, just the physical movement of things is a little more quirky 🙂
I have another two jars on the way. One for Reconfigure and one for Roisin herself.
I am sure she will change her own twitter icon from the regular jar to one of these later as @axelweight Yes she does have a Twitter account, she had to otherwise she would not have been able to accidentally Tweet “ls -l” and get introduced to the World changing device @RayKonfigure would she?
All this interweaving of tech and experience, in this case related to the books, is what I do and have always done. I hope my ideas are inspirational to some, and one day obvious to others. I will keep trying to do the right thing, be positive and share as much as possible.
I am available to talk through any opportunities you have, anytime. epredator at feedingedge.co.uk or @epredator
Finally, last but not least, I have to say a huge thank you to my wife Jan @elemming She has the pressure of the corporate role, one that she enjoys but still it is the pressure. She is the major breadwinner. You can imagine how many 99p books you have to sell make any money to pay anything. She puts up with the downs whilst we at for the ups. Those ups will re-emerge, this year has shown that too me. No matter how bleak it looks, something happens to offer hope. I have some new projects in the pipeline, mostly speculative, but with all these crazy ideas buzzing around something will pop one day.
As we say in Choi Kwang Do – Pil Seung! which means certain victory. Happy lucky 7th birthday Feeding Edge 🙂
Using the Real World – IoT, WebGL, MQTT, Marmite, Unity3d and CKD
All the technology and projects I have worked on in my career take what we currently have at the moment and create or push something further. Development projects of any kind will enhance or replace existing systems or create brand new ones. A regular systems update will tweak and fix older code and operations and make them fresher and new. This happens even in legacy systems. In both studying and living some of the history of our current wave of technology, powered by the presence of the Internet, I find it interesting to reflect of certain technology trajectories. Not least to try and find a way to help and grow this industries, and with a bit of luck actually get paid to do that. I find that things finding out about other things is fascinating. With Predlet 2.0 birthday party we took them all Karting. There was a spare seat going so I joined in. The Karts are all instrumented enough that the lap times are automatically grabbed as you pass the line. Just that one piece of data for each Kart is then formulated and aggregated. Not just with your group, but with the “ProSkill” ongoing tracking of your performance. The track knows who I am now I have registered. So if I turn up and rice again it will show me more metrics and information about my performance, just from that single tag crossing the end of lap sensor. Yes that IoT in action, and we have had that for a while.
The area of Web services is an interesting one to look at. Back in 1997, whilst working on very early website for a car manufacturer, we had a process to get to some data about skiing conditions. It required a regular CRON job to be scheduled and perform a secure FTP to grab the current text file containing all the ski resorts snowfall, so that we could parse it and push it into a form that could be viewed on the Web. i.e. it had a nice set of graphics around it. That is nearly 20 years ago, and it was a pioneering application. It was not really a service or an API to talk to. It used the available automation we had, but it started as a manual process. Pulling the file and running a few scripts to try and parse the comma delimited data. The data, of course, came from both human observation and sensors. It was collated into one place for us to use. It was a real World set of measurements, pulled together and then adjusted and presented in a different form over the Internet via the Web. I think we can legitimately call that an Internet of Things (IoT) application?
We had a lot of fancy and interesting projects then, well before their time, but that are templates for what we do today. Hence I am heavily influenced by those, and having absorbed what may seem new today, a few years ago, I like to look to the next steps.
Another element of technology that features in my work is the ways we write code and deploy it. In particular the richer, dynamic game style environments that I build for training people in. I use Unity3d mostly. It has stood the test of time and moved on with the underlying technology. In the development environment I can place 3D objects and interact with them, sometimes stand alone, sometimes as networked objects. I tend to write in C# rather than Javascript, but it can cope with both. Any object can have code associated with it. It understands the virtual environment, where something is, what it is made of etc. A common piece of code I use picks one of the objects in the view and then using the mouse, the virtual camera view can orbit that object. It is an interesting feeling still to be able to spin around something that initial looks flat and 2D. It is like a drones eye view. Hovering or passing over objects.
Increasingly I have had to get the Unity applications to talk to the rest of the Web. They need to integrate with existing services, or with databases and API’s that I create. User logons, question data sets, training logs etc. In many ways it is the same as back in 1997. The pattern is the same, yet we have a lot more technology to help us as programmers. We have self defining data sets now. XML used to be the one everyone raved about. Basically web like take around data to start and stop a particular data element. It was always a little to heavy on payload though. When I interacted with the XML dat from the tennis ball locations for Wimbledon the XML was too big for Second Life to cope with at the time. The data had to be mashed down a little, removing the long descriptions of each field. Now we have JSON a much tighter description of data. It is all pretty much the same of course. An implied comma delimited file, such as the ski resort weather worked really well, if the export didn’t corrupt it. XML version would be able to be tightly controlled and parsed in a more formal language style way, JSON is between the two. In JSON the data is just name:value, as opposed to XML
Unity3d copes well with JSON natively now. It used to need a few extra bits of code, but as I found out recently it is very easy to parse a web based API using code and extra those pieces of information and adjust the contents of the 3d Environment accordingly. By easy, I mean easy if you are a techie. I am sure I could help most people get to the point of understanding how to do this. I appreciate too that having done this sort of thing for years there is a different definition of easy.
It is this grounding in real World pulling info data and manipulating it, from the Internet and serving it to the Web that seems to be a constant pattern. It is the pattern of IoT and of Big Data.
As part of the ongoing promotion of the science fiction books I have written I created a version of the view Roisin has of the World in the first novel Reconfigure. In that she discovers and API that can transcribed and described the World around her.
This video shows a simulation of the FMM v1.0 (Roisin’s application) working as it would for her. A live WebGL version that just lets you move the camera around to get a feel for it is here.
WebGL is a new target that Unity3d can publish too. Unity used to be really good because it had a web plugin that let us deploy applications, rich 3d ones, to any web browser not just build for PC, mac and tablets. Every application I have done over the past 7 years has generally had the web plugin version at its core to make life easier for the users. Plugins are dying and no longer supported on many browsers. Instead the browser has functions to draw things, move things about on screen etc. So Unity3d now generates the same thing as the plugin, which was common code, but creates a mini version for each application that is published. It is still very early days for WebGl, but it is interesting to be using it for this purpose as a test and for some other API interactions with sensors across the Web.
In the story, the interaction Roisin starts as a basic command line ( but over Twitter DM), almost like the skiing FTP of 1997. She interrogates the API and figures out the syntax, which she then builds a user interface for. Using Unity3d of course. The API only provides names and positions of objects, hence the cube view of the World. Roisin is able to move virtual objects and the API then, using some Quantum theory, is able to move the real World objects. In the follow up, this basic interface gets massively enhanced, with more IoT style ways of interacting with the data, such as with MQTT for messaging instead of Twitter DM’s as in the first book. All real World stuff, except the moving things around. All evolved through long experience in the industry to explain it in relatively simple terms and then let the adventure fly.
I hope you can see the lineage of the technology in the books. I think the story and the twists and turns are the key though. The base tech makes it real enough to start to accept the storyline on top. When I wrote the tech parts, and built the storyboard they were the easy bits. How to weave some intrigue danger and peril in was something else. From what I have been told, and what I feel, this has worked. I would love to know what more people think about it though. It may work as a teaching aid for how the internet works, what IoT is etc for any age group, from schools to boardroom? The history and the feelings of awe and anger at the technology are something we all feel at some point with some element of out lives too.
Whilst I am on real World though. One of the biggest constants in Roisin’s life is the like it or love it taste of Marmite. It has become, through the course of the stories, almost a muse like character. When writing you have to be careful with real life brands. I believe I have used the ones I have in these books as proper grounding with the real World. I try to be positive about everyone else products, brands and efforts.
In Cont3xt I also added in some martial arts, from my own personal experience again, but adjusted a little her and there. The initial use of it in Cont3xt is not what you might think when you hear martial art. I am a practitioner of Choi Kwang Do, though I do not specially call any of the arts used in the book by that name as there are times it is used aggressively, not purely for defence. The element of self improvement is in there, but with a twist.
Without the background in technology over the years and the seeing it evolve and without my own personal gradual journey in Choi Kwang Do, I would not have had the base material to draw upon, to evolve the story on top of.
I hope you get a chance to read them, it’s just a quick download. Please let me know what you think, if you have not already. Thank you 🙂
Making Movies – Flying around in Unity3d
Yes, ok, my latest obsession of writing #Reconfigure might make a greta TV series or Movie but that’s not what this is about. Instead it is about another set of skills I seem to be putting into practice.
Before the I spent a day editing up the school end of year video. I didn’t shoot it, but I did use al the various messing around and comedy moments from the teachers and staff to put a montage together. It had a story, some pace, a step change or two. It was great fun. I have now been asked to edit a few more things together. I like editing moving images, it is fun. Though I am not charging for this is just to help people out.
At the same time I am in the middle of a little project to try and jazz up a regular presentation in the corporate space. I demoed a little virtual fly through a building, past a couple of characters and onto an image using Unity3d. It turned into a slightly more complex epic but non the less I am using lots of varied skills to make it work. The pitch will be pre canned but I have built the Unity environment so that it runs on rails, but live. I have c# code in there to help do certain things like play video textures, animation controllers and even NavMesh agents to allow a character to walk around to a particular room and lots of flying cameras. I had used the first few things a lot. It is stock Unity. However I had not really had a chance to use Unity animations. All my animations had been either remade FBX files or ones that I created in Cheetah3D. Once you imported an animation like that it was then sort of locked in place.
However, Unity3d has its own animation dope and curve sheets, and its really handy.
The animation Tab brings a different timeline. It is different to the Animation Controller screen that lets you string animation together and create transitions. e.g. jump or run from a walk.
The animation tab lets you select an object in the scene and create a .anim for you. It then by default adds that anim to an animation controller and adds that to the object.
The record mode then allows you to expand any or all of the properties of the object, usually transform position and rotation, but it can be colour, visibility, basically anything with a parameter. If you move the timeline and then move the object, like a camera for instance, it creates the key frames for the changed parameters. It removes any of the awkward importing of paths and animation ideas from another tool. It is not great for figures and people. They are still best imported as skinned meshes with skeletons and let the import deal with all the complexity so you can add any stock animations to them. Whoever for a cut scene, or zoom past to see a screen on a wall it works really well.
I have written a lot of complicated things in Unity3d, but never bothered using this part. It is great to find it working, and doing exactly what I needed it to do for my cinematic cut scenes.
You can have things zooming and spinning around a known path in no time. You have to break it down a little otherwise it resolves the easiest path through a wall or something, but it seems very robust to edit it afterwards.
The only pity is that its not possible to lock a camera preview window open in stock Unity3d when selecting another object. Having got a camera pan just right its trick if you was to move an object in the camera view. With camera selected you see it’s preview, with the object selected you see it. Never got around that yet, just end up with trial and error.
No matter, it works, and its awesomely useful.
Return of the camp fire – Blended Reality/Mixed Reality
Back in 2006 when we all got into in the previous wave of Virtual Reality (note the use of previous not 1st) and we tended to call them virtual worlds or the metaverse we were restricted to online presence via a small rectangular screen attached to a keyboard. Places like Second Life become the worlds that attracted the pioneers, explorers and the curious.
Several times recently I have been having conversations, comments, tweets etc about the rise of the VR headsets Oculus Rift et al, AR phone holders like Google Cardboard, MergeVR and then the mixed reality future of Microsoft Hololens, and Google Magic Leap.
In all the Second Life interactions and subsequent environments it has always really been about people. The tech industry has a habit of focusing on the version number, or a piece of hardware though. The headsets are an example of that. There may be inherent user experiences to consider but they are more insular in their nature. The VR headset forces you to look at a screen. It provides extra inputs to save you using mouse look etc. Essentially though it is an insular hardware idea that attempts to place you into a virtual world.
All out previous virtual world exploration has been powered by the social interaction with others. Social may mean educational, may mean work meetings or may mean entertainment or just chatting. However it was social.
Those of us who took to this felt as we looked at the screen we were engaged and part of the world. Those who did not get what we were doing only saw funny graphics on the screen. They did not make that small jump in understanding. There is nothing wrong with that, but that is essential what we were all trying to get people to understand. A VR headset provides a direct feed to the eyes. It forces that leap into the virtual, or at least brings it a little closer. Then ready to engage with people. Though at the moment all the VR is engaging with content. It is still in showing off mode.
We would sit around campfires in Second Life. In particular Timeless Prototype’s very clever looking one that adds seats/logs per person/avatar arriving. Shared focal points of conversation. People gathering looking at a screen in front of them but being there mentally. The screen providing all the cues and hints to enhance that feeling. A very real human connection. Richer than a telephone call, less scary than a video conference, intimate yet stand offish enough to meet most peoples requirements if they gave it a go.
It is not such a weird concept as many people get lost in books, films and TV shows. They may be immersed as a viewer not a participant or they may be empathising with characters. They are still looking at an oblong screen or piece of paper and engaged.
With a VR headset, as I just mentioned, that is more forced. They will understand the campfire that much quicker and see the people and the interaction. There is less abstraction to deal with.
The rise of the blended and mixed reality headset though change that dynamic completely. Instead they use the physical world, one that people already understand and are already in. The campfire comes to you. Wherever you are.
This leads to a very different mental leap. You can have three people sat around the virtual campfire in their actual chairs in an actual room. The campfire is of course replaceable with any concept, piece of information, simulated training experience. Those people each have their own view of the world and of one another but its added to with a new perspective, they see the side of the campfire they would expect.
It goes further though, there is no reason for the campfire not to coexist is several physical spaces. I have my view of the campfire from my office, you from yours. We can even have a view of one another as avatars placed into our physical worlds, or not. It’s optional.
When just keeping with that physical blended model it is very simple for anyone to understand and start to believe. Sat in a space with one another sharing an experience, like a campfire is a very human very obvious one. For many that is where this will sort of end. Just run of the mill information, enhanced views, cut away engineering diagrams, point of view understanding etc.
The thing to consider, for those who already grok the VW concept is that just as in Second Life you can move your camera around, see things from any point of view, for the other persons point, from a birds eye view, from close in or far away, the mixed reality headsets will still allow us to do that. I could alter my view to see what you are seeing on the other side of the camp fire. That is the star of something much more revolutionary to consider whilst everyone catches up the fact that this is all about people not just insular experiences.
This is the real metaverse, a blend of everything anywhere and everywhere. Connecting us as people, our thoughts dreams and ideas. There are huge possibilities to explore and more people will once that little hurdle is jumped to understand it’s people, its always been about people.
Microsoft Minecraft, Space and Semantic Paint
It is interesting looking, spotting, patterns and trends and extrapolating the future. Sometimes things come in and get focus because they are mentioned a lot on multiple channels and sometimes those things converge. Right at the moment it seems that Microsoft are doing all the right things in the emerging tech space, game technology and general interest space.
Firstly has been the move my Microsoft to make Minecraft even easier to use in schools.
They are of course following on and officially backing the sort of approach the community has already taken. However a major endorsement and drive to use things like Minecraft in education is fantastic news. The site is education.minecraft.net
And just to resort to type again, this is, as the video says, not about goofing around in a video game. This is about learning maths, history, how to communicate online. These are real metaverse tools that enable rich immersive opportunities, on a platform that kids already know and love. Why provide them with a not very exciting online maths test when they can use multiple skills in an online environment together?
Microsoft have also supported the high end sciences. Announcing teaming up with NASA to use the blended/mixed/augmented reality headset Hololens to help in the exploration of Mars. This is using shared experience telepresence. Bringing mars to the room, desk, wall that several people are sharing in a physical space. The Hololens and the next wave of AR are looking very exciting and very importantly they have an inclusion approach to the content. Admittedly each person needs a headset, but using the physical space each person can see the same relative view of the data and experience as the rest of the people. The don’t have to of course :), there is not reason for screens not to show different things in context for each user. However standard VR is in immersed experience without much awareness of others. A very important point I feel.
Microsoft have also published a paper and video about using depth scanning and live understanding and labelling of real world objects. Something that things like Google Tango will be approaching.
Slightly better than my 2008 attempt (had to re-use this picture again of the Nokia N95)
The real and virtual are becoming completely linked now. Showing how the physical universe is actually another plane alongside all the other virtual and contextual metaverses.
It is all linked and not about isolation, but understanding and sharing of information, stories and facts.
Alone in a crowd – RPGs and MMORPGs
Free roaming role playing games are one of the most intriguing style of games and experiences to engage with IMHO. I use the word experience, not just games, in order to encapsulate the social environments and metaverse’s such as Second Life, Hi Fidelity et al. However I also did not just restrict that first sentence to include online massively multiplayer/multiuser online. There have been two significant release the past few weeks that are both huge expansive free roaming role playing games The Witcher III and Elder Scrolls online – Tamriel Unlimited. . I have been struck by a few experiences that I have had in these environments. Ones that reach past just playing a game or getting some points/gold/kudos/screenshots.
If you are not a gamer, or don’t engage with these sorts of things they may seem almost identical. They are certainly in a genre and have lots in common.
Witcher III – Xbox one
Elder Scrolls – Xbox One
As you can see in both you are able to take a break from any action and just appreciate the environment. This is not intended to be a graphic comparison though the slightly different styles, yet still aiming for the “real” is noticeable in these two examples.
Witcher III is a single player game. It is a story driven plot. A very long story it would seem too. Like all role playing games it rewards your play with new skills, levels and abilities as you level up. You gradually unlock the very expansive world with all its challenges and new types of monsters and bad guys as you progress through the main storyline. This generally involves traversing the world searching and fighting in various ways. It has a 3rd person style of free form combat. You choose the type of weapon and various potions and armour to give you a fighting chance against increasingly higher level creatures. You can just wander off the plot and the beaten track and see what you bump into. However this particular RPG reminds you constantly that whilst you might feel you have learned a lot there is still more levelling up to do as you arrive at a place in a world full of things that can cut you down with a single strike. Level systems in RPG’s generally appear as numbers on the display. When you have spent 20 hours playing and just reached level 7 and you wander into a clearing with a large beast that says level 35, where level progression is non linear and requires exponentially more effort and time per level you know you are in trouble. Witcher reminds you that no matter how heroic and skilled you thing you are, there is always something bigger and better. It has a grittiness to the entire game that brings a sense of foreboding, yet it is still entertaining. However you are very much alone and left to your own devices.
Elder Scrolls Online – Tamriel Unlimited is a very different feel and a very different experience. It is Skyrim (the one player RPG from a few years ago), reborn into a persistent Massive Multiple Player Online experience. Skyrim had been noted for it’s random generation of dungeons combined with its epic land size and the variation of things that you ended up doing in the virtual environment. Elder Scrolls has both a 3rd person and a 1st person view. The latter lets you see the world, and just your hands/ weapons. It lets you use each and on its own, weapon and shield combinations. The weapon can be ranged magic in one hand and a sword in the other. You travel the world finding missions from non player characters. Most of these involve collecting, running to and fro and just like the Witcher III putting things together and crafting items you need, potions, armour, magic, food etc.
So unlike Witcher III your character is in the environment with lots of other people. Just like the other giants of the MMORPG genre, World Of Warcraft and Eve Online. It is quite unusual for this sort of genre to make it to a console. Usually these platforms get multiplayer shooters where 16-32 people battle for a few minutes in an arena that is then destroyed and reformed for the next battle.
Here we have a world that we all share, all the time. Of course the technical detail of which server, or how anyone is actually in the “same” place is obscured and not entirely relevant to the experience. Also the world itself is generally a locally loaded place. The servers are there to broker player position, communication and hold certain shared features in the experience, but it is not really a full persistent world.
I have found it works really well as a game to be wandering around, getting engrossed in each task, gradually levelling up, choosing skill tree paths, working out what to carry, sell, deconstruct and to craft. Lots of the missions from the non player characters are actually there to help you learn the very complex system of crafting as much as forwarding any plot line points. Choosing missions chooses various guilds and organisations to which you belong.
I called this post “alone in the crowd” because it is a particular feature of this style of environment that you do see lots and lots of other people, milling around playing their own missions, but you do not have to interact with them if you don’t want to. Though sometimes it is unavoidable, or required to work in a team.
This is by no means a crowd shot! but the character not the right is just running off somewhere. They ran past me, I paid no attention to them, nor them to me other than we just both know we are part of the human noise of this virtual city.
Most things are not restricted based on numbers in the free roam areas. For example the blacksmith station is a object you have to walk up to, select and then take part in an inventory management/crafting dialogue. The things you are doing are entirely related to you and your experience, but we all have to gather around the anvil in order to do this. This is good when there are only a few people, but when it’s busy the room is full of lots of gormless characters standing around.
Passing one another in the world feels like something we normally do, it adds to the sense of a shared experience, but all standing around staring into space, or watching people traverse the land mostly by jumping and bounding around in order to get for a to b quickly starts to look like a complete mess of uncoordinated activity. Which of course, it is 🙂
This is a feature of most virtual world online experiences. The only time it is not is when people are more seriously role playing and rules are set to preserve the experience. This happens in some areas in Second Life where you are encouraged to be wearing the right clothes, speaking the right way to join in with the entertainment of acting. Rather like not turning at a real life roman re-enactment on a segway wearing a stetson.
That is not to detract from the game, or games like it. This is a genre where stuff like this has to happen due to the way the tech works, but also the way people work. It is funny, but it can be a jolt in the immersion.
Where this sharing of space does work though is in battles with things in the environment and things in it. You can be wandering around exploring and then be set upon by a pack of wolves. When you start fighting it becomes “your” battle. However other people passing by can also see and join in with that battle, or choose it ignore it. Those wolves become a shared experience. There are a lot more bad guys than wolves but it’s an easy example to use. For a short period of time, you and the others who just happen to be there at the time are no longer alone. You are working with the same problem. Now this could just be thought of as a few experience points of resources to add, share split, ignore but it has all sorts of extra dynamics.
If you see someone in trouble, do you help ? Does your helping actually hinder and annoy them? as they were challenging themselves and their character.
Sometimes you just have to join in in order to be able to get the things you need to proceed. In one particular example the bits of a monster were needed in order to escape an area. If someone else kills the monster before you get any sort of hit in then you are not credited with any part of the kill so do get the bits you need. This meant waiting, or searching for another one of the same type, waiting for a re-spawn and getting in there before the rest of the crowd. Game design and rules may have altered how many people might be competing for this single resource, and I did escape, but it was an interesting co-opetition dynamic.
This led also to another thought on timelines, on the immersive experience and how we are often reminded of our place in these virtual worlds. Having worked towards a small piece of the plot and defeated a particular boss, whereby a slight levelling occurred, new kit and a small buzz of excitement at having completed something, whilst standing near the same spot taking stock a new crowd arrived. Of in the distance I heard the bad guy arrive again, go through the same “No mr bond I expect you to die” style of dialogue. I watched from the distance as my fellow travellers and gamers fought the same battle, but in a different way. In my timeline, this battle was over. I was stood in what was attempting to be a coherent virtual experience being reminded I was not. It is not a problem, as such, but it is a thing.
It is reminder and a breaking of the story. Yet I think there may be something in this kind of feeling of immersion and then shocking back into reminders it is not real. It is an accident of the technology restrictions that cause this to happen. It is a technique used in other media when the action breaks the plane to the fourth wall. When a stage actor talks to the audience when previously the audience were passive. Some of the most chilling of these have been Kevin Spacey in House of Cards. The very occasional, to the point you think they have forgotten to write any in, turning to the audience and engaging in dialogue is very special. One of the most memorable being “Oh did you think I had forgotten about you?” still gives me goosebumps thinking about it.
It should not matter if we choose to form clans and take on multiplayer missions, or just act on our own in these worlds. The very mechanism we are using has the ability to reach back out to us and surprise us. I have felt this in games just a few times. It may be a party trick in some cases but reaching out to the human, through the role play, through the immersion may just be the most memorable and entertaining thing a virtual world can do.
Untethering Humans, goodbye screens
We are on the cusp of a huge change in how we as humans interact with one another, with the world and with the things we create for one another. A bold statement, but one that stands up, I believe, by following some historical developments in technology and social and work related change.
The change involves all the great terms, Augmented Reality, Virtual Reality, Blended Reality and the metaverse. It is a change that has a major feature, one of untethering, or unshackling us as human beings from a fixed place or a fixed view of the world.
Here I am being untethered from the world in a freefall parachute experience, whilst also being on TV 🙂
All the great revolutions in human endeavour have involved either transporting us via our imagination to another place or concept, books, film, plays etc. or transporting us physically to another place, the wheel, steam trains, flight. Even modern telecommunications fit into that bracket. The telephone or the video link transport us to a shared virtual place with other people.
Virtual worlds are, as I may have mentioned a few times before, ways to enhance the experience of humans interacting with other humans and with interesting concepts and ideas. That experience, up to know has been a tethered one. We have evolved the last few decades becoming reliant on rectangular screens. Windows on the world, showing us text, images, video and virtual environments. Those screens have in evolved. We had large bulky cathode ray tubes, LED, Plasma, OLED and various flat wall projectors. The screens have always remained a frame, a fixed size tethering us to a more tunnel vision version. The screen is a funnel through which elements are directed at us. We started to the untethering process with wi-fi and mobile communications. Laptops, tablets and smartphones gave us the ability to take that funnel, a focused view of a world with us.
More recent developments have led to the VR headsets. An attempt to provide an experience that completely immerses us by providing a single screen for each eye. It is a specific medium for many yet to be invented experiences. It does though tether us further. It removes the world. That is not to say the Oculus Rift, Morpheus and HTC Vive are not important steps but they are half the story of untethering the human race. Forget the bulk and the weight, we are good at making things smaller and lighter as we have seen with the mobile telephone. The pure injection of something into our eyes and eyes via a blinkering system feels, and is, more tethering. It is good at the second affordance of transporting is to places with others, and it is where virtual world and the VR headsets naturally and obviously co-exist.
The real revolution comes from full blended reality and realities. That plural is important. We have had magic lens and magic mirror Augmented Reality for a while. Marker based and markerless ways to use one of these screens that we carry around or have fixed in out living rooms to show us digital representations of things places in out environment. They are always fun. However they are almost always just another screen in our screens. Being able to see feel and hear things in our physical environment wherever we are in the world and have them form part of that environment truly untethers us.
Augmented Reality is not new of course. We, as in the tech community, has been tinkering with it for years. Even this mini example on my old Nokia N95 back in 2008 starts to hint at the direction of travel.
The devices used to do this are obviously starting at a basic level. Though we have the AR headsets of Microsoft Hololens, Google’s Magic Leap to start pushing the boundaries. They will not be the final result of this massive change. With an internet of things world, with the physical world instrumented and producing data, with large cloud servers offering compute power at the end of an wireless connection to analyse that data and to be able to visualize and interact with things in our physical environment we have a lot to discover and to explore.
I mentioned realities, not just reality. There is no reason to only have augmentation into a physical world. After all if you are immersed in a game environment or a virtual world you may actually choose, because you can, to shut out the world, to draw the digital curtains and explore. However, just as when you are engaged in any activity anywhere, things come to you in context. You need to be able to interact with other environments from which ever one you happen to be in.
Take an example, using Microsoft Hololens, Minecraft and Skype. In todays world you would have minecraft on you laptop/console/phone be digging around, building sharing the space with others. Skype call comes in from someone you want to talk to. You window away from Minecraft and focus on Skype. It is all very tethered. In a blended reality, as Hololens has shown you can have Minecraft on the rug in front of you and skype hanging on the wall next to the real clock. Things and data placed in the physical environment in a way that works for you and for them. However you may want to be more totally immersed in Minecraft and go full VR. If something can make a small hole in the real world for the experience, then it can surely make an all encompassing hole, thus providing you with only Minecraft. Yet, if it can place Skype on your real wall, then it can now place it on your virtual walls and bring that along with you.
This is very much a combination that is going to happen. It is not binary to be either in VR or not, in AR or not. It is either, both or neither.
It is noticeable that Facebook, who bought heavily into Oculus Rift have purchased Surreal Vision last month who specialize in using instrumentation and scanning kit to make sense of the physical world and place digital data in that world. Up until now Oculus Rift, which has really led the VR charge since its kickstarter (yes I backed that one!) has been focussed on the blinkered version of VR. This purchase shows the intent to go for a blended approach. Obviously this is needed as otherwise Magic Leap and Hololens will quickly eat into the Rifts place in the world.
So three of the worlds largest companies, Facebook, Google and Microsoft have significant plays in blended reality and the “face race” as it is sometimes called to get headsets on us. Sony have Project Morpheus which is generally just VR, yet Sony have had AR applications for many years with PS Move.
Here is Predlet 2.0 enjoying the AR Eyepet experience back in 2010 (yes five years ago !)
So here it is. We are getting increasingly more accurate ways to map the world into data, we have world producing IOT data streams, we have ever increasing ubiquitous networks and we have devices that know where they are, what direction they are facing. We have high power backend servers that can make sense of our speech and of context. On top of that we have free roaming devices to feed and overlay information to us, yes a little bit bulky and clunky but that will change. We are almost completely untethered. Wherever we are we can experience whatever we need or want, we can, in the future blend that with more than one experience. We can introduce others to our point of view or keep our point of view private. We can fully immerse or just augment, or augment our full immersion. WE can also make the virtual real with 3d printing!
That is truly exciting and amazing isn’t it?
It makes this sort of thing in Second Life seem an age away, but it is the underpinning for me and many others. Even in this old picture of the initial Wimbledon build in 2006 there is a cube hovering at the back. It is one of Jessica Qin’s virtual world virtual reality cubes.
That cube and other things like it provided an inspiration for this sort of multiple level augmentation of reality. It was constrained by the tethered screen but was, and still is, remarkable influential.
Here my Second Life avatar, so my view of a virtual world is inside 3d effect that provides a view, a curved 3d view not just a picture, or something else. It is like the avatar is wearing a virtual Oculus rift it that helps get the idea across.
This is from 2008 by way of a quick example of the fact even then it could work.
So lets get blending then 🙂
Hololens – Now this is cool progress!
Yesterday Microsoft had a big series of announcement at its Build 2015 conference. The most exciting demonstration, IMHO, was on stage with Hololens. In this video they have a camera rendering the cameraman point of view of the digital content attached to real space, while the demo person is wearing their Hololens and operation from their perspective.
Taking applications that are floating in your heads up display, and attaching them to real world was, tables etc is very clever. Interacting with those objects even more so. The part at the end of this clip with the video screen and resizing it…. well who needs a TV now then?
This sort of blended reality (yes that term again I keep using as this is so much more than just augmentation of reality) creates a shared space that in this case the camera viewer is also allowed to see. However, as I mentioned in my last post there is no reason for the view to be the same or shared. Both modes can work. We can both be watching the same film but my weather desk object might be showing a completely different place to yours. Or we can how to completely different self contained shards of rooms yet in the same place. Throw in physical creation of objects in 3d printing or force feedback haptics and we have one heck of an environment to explore.
Microsoft have published a very slick video too of the hardware
They obviously have gone first on this tech but Google have Magic Leap waiting in the wings. There is a race on.
Whilst clearly many of us, not matter how much we have been around this sort of tech and around virtual worlds, are not going to just get sent a Hololens to build with we can still explore some of the principles using old fashioned tech like a smartphone and MergeVR/Cardboard etc and packages like Vuforia. Though Microsoft…. If you want to send me one or sign up a tech evangelist for this stuff (though it will sell itself) I am available anytime!
We have had AR to experiment with for a while but this wave is true 3d interaction with a markerless environment. Back in 2007! I wondered what it would be like to put an AR marker as a texture into a virtual world (in this case Second Life) and then point a camera with an AR application running at the screen that was rendering SL.
The result was that the Ms Pacman in this slightly blurry picture, is not visible to my epredator avatar in Second Life. That client only sees the marker. Yet as we pan the camera around or look at the marker the second application, which is an AR magic lens, renders Ms PacMan.
Blended reality does not have to be solely one layer into the physical world. It is the easiest to demonstrate but we have so many more choices to explore and opportunities.
I made a little video to show the basic Vuforia demos in Unity3d that are marker based. This stuff just works these days. It is less faffing about than it used to be. It is fantastic to see everything get so accessible and slick.
This is a tipping point. Virtual content, virtual environments, virtual worlds get even more engaging with the more direct natural inputs we can make to our body. Full VR is able to remove the world and drop you somewhere that you can’t be or that doesn’t exist. Blended Reality is able to bring things that don’t exists to the space you inhabit. The interaction metaphors are going to be interesting too. Do you render a virtual waste bin to throw throw applications into when done or do you map that to a small crackling virtual fireplace that sits in the actual fireplace in your living room?
I am going to stop listing things as this will go on for a while. Time to go and build some things to share later.
What’s the point of Goat Simulator?
Firstly, this is not me asking the question. It was something that just happened and serendipity took over to explain. For this of you who have not experienced goat simulator on any platform it is… well… a… Goat Simulator. In fact it is a large physics sandbox were you happen to gallop around as a rather difficult to control goat, jumping, bashing, licking and rolling around.
It makes no bones about its perfection, or lack of it. The physics is good but the collision detection is wonky at times, but thats its charm. It has many elements you would find in skating game. Points systems, achievements and just a little tricky on the controls.
I bough GS a while back on the Mac when we were on holiday and we had a joke about the menacing look from the goats. Now it is there on Xbox One for only £7.99 so well worth it as its hilarious.
Now many hardcore gamers may be busy on the PS4 trying to deal with Bloodborne. Which is, IMHO, just too damn hard despite being fantastic.
GS is just fun, its a sandbox game, you make of it what you want, find your own paths and challenges.
Now @elemming was sitting playing some resource farming game on the ipad and happened to say “what’s the point of goat simulator?”. Just at that point this happened. I butted a pedastrian who ran off and… well watch the video. It made elemming bust into laughter about 30 seconds after looking up and saying “Whats the point….”
Well thats the point 🙂 it’s just fun, with some challenges.
Games can be that, or games can be very serious and intensely stressful. They all work, they are all worth exploring and they all fit various situations. The fact this is a virtual world of course fits with what I do for a living. However it’s all good