Yesterday Microsoft had a big series of announcement at its Build 2015 conference. The most exciting demonstration, IMHO, was on stage with Hololens. In this video they have a camera rendering the cameraman point of view of the digital content attached to real space, while the demo person is wearing their Hololens and operation from their perspective.
Taking applications that are floating in your heads up display, and attaching them to real world was, tables etc is very clever. Interacting with those objects even more so. The part at the end of this clip with the video screen and resizing it…. well who needs a TV now then?
This sort of blended reality (yes that term again I keep using as this is so much more than just augmentation of reality) creates a shared space that in this case the camera viewer is also allowed to see. However, as I mentioned in my last post there is no reason for the view to be the same or shared. Both modes can work. We can both be watching the same film but my weather desk object might be showing a completely different place to yours. Or we can how to completely different self contained shards of rooms yet in the same place. Throw in physical creation of objects in 3d printing or force feedback haptics and we have one heck of an environment to explore.
Microsoft have published a very slick video too of the hardware
They obviously have gone first on this tech but Google have Magic Leap waiting in the wings. There is a race on.
Whilst clearly many of us, not matter how much we have been around this sort of tech and around virtual worlds, are not going to just get sent a Hololens to build with we can still explore some of the principles using old fashioned tech like a smartphone and MergeVR/Cardboard etc and packages like Vuforia. Though Microsoft…. If you want to send me one or sign up a tech evangelist for this stuff (though it will sell itself) I am available anytime!
We have had AR to experiment with for a while but this wave is true 3d interaction with a markerless environment. Back in 2007! I wondered what it would be like to put an AR marker as a texture into a virtual world (in this case Second Life) and then point a camera with an AR application running at the screen that was rendering SL.
The result was that the Ms Pacman in this slightly blurry picture, is not visible to my epredator avatar in Second Life. That client only sees the marker. Yet as we pan the camera around or look at the marker the second application, which is an AR magic lens, renders Ms PacMan.
Blended reality does not have to be solely one layer into the physical world. It is the easiest to demonstrate but we have so many more choices to explore and opportunities.
I made a little video to show the basic Vuforia demos in Unity3d that are marker based. This stuff just works these days. It is less faffing about than it used to be. It is fantastic to see everything get so accessible and slick.
This is a tipping point. Virtual content, virtual environments, virtual worlds get even more engaging with the more direct natural inputs we can make to our body. Full VR is able to remove the world and drop you somewhere that you can’t be or that doesn’t exist. Blended Reality is able to bring things that don’t exists to the space you inhabit. The interaction metaphors are going to be interesting too. Do you render a virtual waste bin to throw throw applications into when done or do you map that to a small crackling virtual fireplace that sits in the actual fireplace in your living room?
I am going to stop listing things as this will go on for a while. Time to go and build some things to share later.
Monthly Archives: April 2015
Just Cause 3 – Its been 5 years since ….
It’s been five years since I wrote this post about Just Cause2 in which I mentioned “Sandbox games are the closest to what we see in virtual worlds in crossover terms. Just Cause 2 is a single player game, in part because the destruction you create is really personalising your island. However the balance between “give me something to do” and “I am just going to find something to do” may be something the VW providers and content creators need to consider in helping people find a narrative, either social or business related.”
Well I stand by that. The mix of ready made tasks, ongoing narrative plot live and freedom to find your own things to do as you wander around is a core theme in some of the most enjoyable experiences. Minecraft, Goat Simulator, GTA V Watch Dogs etc.
Now we have the reveal of gameplay for Just Cause 3. Which if nothing else looks fantastically exciting 🙂
Fingers crossed it is as good as the last one, it certainly looks it.
Amiga kickstarter book, breaking mirror worlds & VR
A few days ago my copy of the book “Commodore Amiga: a visual Commpendium book” that I backed in Kickstarter arrived. The book is by Sam Dyer through BitmapBooks. It came with a load of extra goodies from my backing and my name along with my fellow backers vanity printed in the appendix. The only slight problem was that unlike all the other Kickstarter campaigns I wasn’t “allowed” to have a credit as epredator as it made the list look untidy unless we had normal names. That is the authors choice of course 🙂
My computer owning history went ZX81, Commodore 64 then Amiga 500 (and later 1200). The Amiga was 1987 and became my main machine for most of my polytechnic/university time. It caused me to get an overdraft for the first time to buy an external hard drive for a piece of work I was doing (that and to play the later cinemaware games that needed two floppy disk drives to work).
It was the machine I coded my final year project, which was a mix of hardware and software but also had to work on the much larger and more expensive Apollo computers we had.
It is the machine I spent ours with sellotaped together graph paper planning my SimCity builds and mapping Bards Tale Dungeons.
It is also the machine I first experience proper network gaming on with a null modem cable and F/A-18 flight simulator. Not only was that the first proper LAN party gaming but it forged the idea that machines do not have to have a consistent shared view of the world just because they are connected. The F/A-18 simulator let my good friend Wal and I fly around shooting at one another in a shared digital space. It was the early days of having a printer and being able to do “desktop publishing” aka DTP. I even produced a poster for our little event.
When we played we had different terrain packs running on each Amiga as we had different versions of the game. There was no server this was really peer to peer. The terrain was local to each Amiga, but the relative location of one another in that space was shared. Each machine was doing its own collision detection. It meant if I saw mountains I needed to avoid them, yet on the other machine that same local space might be flat desert. We all perceive reality differently anyway, but here we were forced to perceive and act according to whatever the digital model threw at us. In reality we kept to the sky and forcing your opponent into their own scenery was considered unsporting (though occasionally funny and much needed).
This set the precedent for me that whilst mirror worlds, virtual worlds that attempt to be 100% like a real place, have a reason to exist we do not have to play by the same physical rules of time and space in virtual environments.
Other things of note about the Amiga. Well I coded as predator on the Commodore 64 and that moved across to the Amiga too. The e was a later addition on the front by the principles are the same.
My wife also discovered gaming on my Amiga. Getting completely wrapped up and in the zone on Sim City and realising it was 4am. Later it would be Lemmings that caught her attention. Hence she is now elemming on twitter.
The book is full of classic images nearly all of which I have some sort of memory of that is more than yes I recognise that picture.
Games like Alien Breed (a gauntlet like top down shooter) and The Secret Of Monkey Island (a classic point and click humorous adventure) on their own rack up considerable hours of entertainment for very different reasons
Whilst fondly reminiscing and remembering things that impact how I think and work today I was also at the same time in current and future mode. Right next door on the table was my copy of Games(TM)
As I tweeted at the time My life history in 1 picture #nearly #amiga #vr #metaverse.
When we put on a headset, a total immersion one, we get a view of a world that is instantly believable. Something fed directly to our eyes and linked to the direct we are looking becomes a convincing reality. In a shared virtual world we will assume that we are all seeing the same thing. That does not have to be the case, as with the F/A-18 example. We can have different experiences yet share the same world. To help think about that consider the game Battleships. Each player has the same map, the same relative grid references on a piece of paper or on plastic peg board. yet on that map you can only see your own boats and any pieces of information you have gained through playing. When considering a mirror world or a virtual world build it can be harder to consider this. Yet many games and environments already have a little dollop of this behaviour with personal displays of health, ammo, speed etc in a Heads up Display. Those HUDs are an augmented reality display in a virtual world.
When we now consider the blended view headsets like the HoloLens and the MergeVR we are taking the real world as the server in effect. It is a fixed environment. We are then placing and enhancing what we see in that world with 3D overlays. Convincing the viewer the digital stuff is real.
Unlike the F/A-18 terrain the real world is there for each person. If there is a table in the middle of the room, even if you make it look like it is not for a headset wearer with object removal and clever visuals they will still trip over it. However the other way around can make for an interesting dynamic. headset wearers made to think there are obstacles and things in their way that they have to move around, but its different for each headset wearer. Just a little though experiment in perception. I didn’t even throw in anything about 3D printers actually making the new obstacles in the real world. That’s a bit much for a monday morning.
Anyway, the Amiga book is great. It was a fantastic period in games and in home technology, but we have many more exciting times coming.
What’s the point of Goat Simulator?
Firstly, this is not me asking the question. It was something that just happened and serendipity took over to explain. For this of you who have not experienced goat simulator on any platform it is… well… a… Goat Simulator. In fact it is a large physics sandbox were you happen to gallop around as a rather difficult to control goat, jumping, bashing, licking and rolling around.
It makes no bones about its perfection, or lack of it. The physics is good but the collision detection is wonky at times, but thats its charm. It has many elements you would find in skating game. Points systems, achievements and just a little tricky on the controls.
I bough GS a while back on the Mac when we were on holiday and we had a joke about the menacing look from the goats. Now it is there on Xbox One for only £7.99 so well worth it as its hilarious.
Now many hardcore gamers may be busy on the PS4 trying to deal with Bloodborne. Which is, IMHO, just too damn hard despite being fantastic.
GS is just fun, its a sandbox game, you make of it what you want, find your own paths and challenges.
Now @elemming was sitting playing some resource farming game on the ipad and happened to say “what’s the point of goat simulator?”. Just at that point this happened. I butted a pedastrian who ran off and… well watch the video. It made elemming bust into laughter about 30 seconds after looking up and saying “Whats the point….”
Well thats the point 🙂 it’s just fun, with some challenges.
Games can be that, or games can be very serious and intensely stressful. They all work, they are all worth exploring and they all fit various situations. The fact this is a virtual world of course fits with what I do for a living. However it’s all good
VR – Everything old is new again – good!
It is important when getting interested and excited about new things to look at its lineage. Something I often did in my series of articles over the years in Flush Magazine
So with the current rush of Virtual Reality gaming and experiences and the slew of new kit we have some very interesting near off the shelf kit such as this.
*warning it contains violence (see where I am on that here)
However, back in the 90’s we had VR kit like Virtuality (they must be kicking themselves for be too early for the masses. Though thats a curse us early adopters and evangelists have to live with 🙂
The headsets were quite heavy and the container you stood or sat in was not a treadmill but part of the sensing rig.
The graphics may look old fashioned and clunky but they were good experiences. When I was at poly/uni in Leicester getting ready to do a year out this was the company I wanted to go and work for if I had a chance too. They sent to me IBM instead. Though as you can see from the Virtuality company details they had very close ties with IBM. So it was close 🙂 It i also funny how things work out.
Still, it is very cool having all these headset sat around me and some that just work on my portable communication device too 🙂
Over 15 years experience in Virtual Worlds, 30+ in tech – What now?
A few days ago I realised it has been over 9 years since I first publicly blogged about how important I thought the principles of the metaverse and virtual worlds were going to be for both social and business uses. This post, pictured below for completeness was a tipping point for some radical changes in many of our lives as part of Eightbar.
I had been working to that sort of point of understanding though since some very early work with virtual environments and how people get to interact with one another in them around 1999/2000 with SmartVR trying to keep the social bond in our internal web and multi media design group together when we were cast asunder to different locations by business pressures (or bad decisions who knows!). I knew we had to have a sense of one another existence aside from text in emails and instant messages. So we tried to build a merged version of both offices as if they were in one place. The aim to then instrument those with presence and the ability to walk over to someone’s desk and talk. Mirror world, blended reality and even internet of things, Yes I know, a bit before its time! Cue music, “Story of my life” by 1D 🙂
We definitely had a technology and expectation bubble later 2006-2009. However, that, as with all emerging technology is all part of the evolution. The garnet curve et al. What surprises me the most still is that people think when a bubble like that bursts that its all over. That somehow everything that was learned in that time was pointless. “Are virtual worlds still a thing?” etc. I feel for the even earlier pioneers like Bruce Damer, who patiently put up with our ramblings as we all rushed to discover and feel for ourselves those things he already knew.
Increasingly I am talking to new startup ands seeing new activity in the virtual space. The same use cases, the same sparks of creativity that we had in the previous wave(s), the same infectious passion to do something interesting and worthwhile. Sometimes this is somehow differentiated from the last wave of virtual worlds under the heading of virtual reality. The current wave is focussed on the devices, the removal of keyboard and of a fixed screen. The Oculus Rift, HoloLens etc. However, thats a layer of new things to learn and experience on what we have already been through. After all its a virtual world you end up looking at or blending with in a VR headset!
I spend so much time looking forward and extrapolating concepts and ideas it is now very scary to look back and consider the experience I have gathered. The war stories and success stories, the concepts and ideas that I have tried. The emotional impact of virtual worlds. The evolution of the technology and of people’s expectation of that technology. The sheer number of people that have moved around in a 3d environment from an early age with things like Minecraft, who are now about to enter higher education and the workforce.
So I am left in a slightly bemused state as to what to do with this knowledge. With this all going so much more mainstream again I am no longer working in a niche. Do I ply my trade of independent consulting chipping away in odd places and helping and mentoring some of the new entrants in the market or do I try and find a bigger place to spread the word?
At the same time though, knowing lots of things makes you realise how much you don’t know. The imposter’s syndrome kicks in. Surely everyone must know all this stuff by now? It’s obvious and stands up to logical reasoning to try and connect with other people in as rich a way as possible. The network is there, the tech is there, the lineage is there. Though clearly not everyone gets it yet. I often wonder if the biggest naysayers I had to deal with on my journey so far have figured it out yet? It will probably turn out they will be getting rich quick right now whilst I sit and ponder such things.
On the other side of the virtual coin, I know from my martial arts that constant practice and investigation leads to a black belt. In the case of Choi Kwang Do that’s just over 3 years worth. So how many Dan worth of tech equivalent experience does that put me at. 25 years in the industry professionally but 30+ as techie.
I still want to do the right thing, to help others level themselves up. I don’t think I am craving fame and fortune but the ability to share and build is what drives me. If fame and fortune as a spokesperson, evangelist for some amazing idea or TV show reaches that end, then thats fine by me.
I am at a crossroads, my VR headset letting me look in all directions. I see half built roads in many directions. What now? A well funded company that I can help build a great road with, or forge off down one for the other paths seeing what happens on the way.
Of course that makes it seem like there is a clear choice on the plate. I suspect most of the the well funded companies and corporations don’t think they need any help, which is rather where I came in on this!
Needless to say I am always open to conversations, offers, partnerships, patronage, retainers and technology evangelist roles. There must be a slew of investors out there wondering what to put their money into, who need some sagely advice ;). Or that book… there is always that book… (600 images x 1000 words each 600,000 words just on these Second Life experiences. That’s just the ones with online. The offline ones double that!. Not to mention the other places, games, and self built experiences!
I took this photo in April 2006 as part of sharing of our journey
I have always liked a nice greenfield to start building on. Equally that did not build itself, it was a massive shared team experience. No one has all the answers. Some of us are good at helping people find them though.
Right! Can we get on with this now?
Silence isn’t golden – VOIP and Unity3d
Once again I am back on the trail of a good voice solution to use in Unity3d to use in multiple users situations. Having done a good few projects using a variety of solutions, mainly with an adjusted version of the now defunct Uspeak (as in removed from the assets store but replaced with Daiken Forge voice) combined with self hosted Photon server I think it must be time to find a more scalable solution. Very often the pressures of voice lead to a swamping of any server that is also dealing with position and data model synchronisation across the network. This clearly makes sense as there are only a few bytes of message for something moving a long distance, yet an audio stream is a constant collection of data flowing in and out of a server.
Whilst there is huge focus on VR for the eyes, being able to communicate needs more than one channel but voice is one of those.
A voice connection can be 100Kbps per person so without a different tuning or set of priorities the regular multiplayer data messages, even if they are reliable, will get lost on the noise of an open mic.
My projects up to now have ended up with a push to talk or even my scarily complicated dial, ring answer/engaged phone simulator in order to cut down on the amount of voice. Raising a hand to speak works in many formal situations but not in social collaboration.
So whilst we can squeeze voice packets onto the same socket server as the rest of the “stuff” it feels like it should be a separate service and thread.
Oddly there is not a solution just sitting there for Unity3d? It seems that many of the existing voice clients for games and related services just don’t play well in the Unity3d/c# environment.
Unfortunately this seems to be down to the generic nature of Unity3d (which is a major strength) compared with the specifics of PC architecture and also in how to deal with shared memory and pipes.
There are a lot of voice servers out there for games. Mumble/Murmur is a major one. it is open source and appears to bolt on top of many games even supporting a 3d positional audio.
Many games are of course closed code bases. So Mumble has to piggy back on top of the game or have a little bit of help form the game developers to drop some hooks in. The main problem seems to be the use of shared memory versus a more closed boundary of applications. Obviously a shared memory space to communicate can bring problems and hacks. Strangely it was shared memory and various uses of it that let us do the early client server work back in the 90’s scraping from a terminal screen to make a legacy CICs application work with a fancy GUI!. Initially I figured that there would be a c# client for mumble that any of us could drop into Unity3d and away we go. However there are a few unfinished or implementations being attempted. Connecting to mumble and dealing with text is fine but voice and the various implementations of encoding seems to be a sticking point. Mumble also has a web interface implemented through something called ICE which is another Remote Procedure Call set of interfaces. It seems to be again focussed on building with C++ or PHP. Whilst ICE for .Net exists it does not work on all platforms. I am still looking at this though as surely if a web interface works we can get it to work on Unity3d. Of course the open source world is wide and diverse so there is always a chance I am missing something?
If a Mumble, or similar client can exists in a Unity3d context then we have our solution. It is nice to have the entire interface you need inside Unity3d not side loaded or overlayed. However it may be that a solution is to have implement the web interfaces next door to a unity interface and use plugin to webpage communication as the control. This is fraught with errors, browsers, DOM’s, script fails etc. Just a nice drop in mumble client hooked u to the shiny new UI interfaces.
I looked at the others in the area, Teamspeak is one of the biggest but incurs licensing charges so is not very open source minded. Ventrillo and Vivox seem to have fallen away.
Then of course there are the actual phone voip packages and standards. Maybe they offer the answer. I used Freeswitch on Opensim to provide a voice channel. Maybe I need to look at those again.
Or another solution could be trying to persuade the socket server applications like Photon than they need to talk to more than one server. At the moment the assumption is one server connection (though the load balancing may occur further on the server itself). If the client knew to talk to a different configure machine and network that was happy to deal with the firehose of audio but the normal movements talked to the server configure to deal with normal movements then we might have a solution.
Either way, its not straight forward and hasn’t been totally solved yet, or if it has everyone is keep quiet (Yes thats ironic when talking about VOIP!)
Let me know if you do have some great opensource or open sourceable solution.
High Fidelity – A new metaverse – It’s here
A while back the original creator of Second Life (Philip Rosedale) announced a new ground up approach to shared virtual spaces taking in all the latest and greatest in tech. A few days ago I saw an email saying the alpha was live. The trouble was I thought it was an April fool joke. Not that I did;t believe it would come along but the timing was a bit odd.
However…. I registered (having been non the tell me more list since the beginning) downloaded the “interface” and voila….
I did swap my mini robot avatar for a full mean machine one. Though there are a few more human and cartoon ones.
This is an alpha, it is not the final product things will improve but it feels very full on.
There are stacks of developer menu options and full support for all sorts of inputs, kinect etc, and outputs, Oculus Rift etc. There is a stack of documentation and youtube videos on how it works too. All to be expected but nice to see at at alpha stage.
I will of course look at building, scripting and hosting in due course. Javascript appears to be the script language of choice here, but that obviously depends on what library we get to talk to. There seems to be lots of terence to nice industry standard 3d models like FBX etc.
I jumped to another hosted area, the obligatory Sandbox and set my display name to epredator too of course 🙂 I might be new here, but I have been here before too 🙂
Lets see where this exciting development takes us.
Kids,video games, parents and ratings
There has been a lot of press and discussion about schools in the UK writing to parents to warn them about young children playing video games. Also indicating that allowing kids to play games rated 18 when they are underage is a sign of neglect and will lead to the parents being reported to police. The BBC covered it here
As a parent, as a school governor, an instructor and long term gamer it seems that this raises a few much more complex issues.
An 18 certificate on anything helps flag up content of a nature regarded as adult. However it does not indicate which of the adult themes it contains. You may notice on films on TV it will say its 18 but whether it contains violence, drug use, nudity, etc. That informs the viewer if they are sensitive or prefer not to watch such things to avoid it.
A parent should know what is in a product that they let their kids loose on. The 18 should be a big flag warning that there may be something in there that is not appropriate. However not all parts of all games are really 18 worthy. It is a marketing ploy to get more adult gamers to slip a few things in sometimes to warrant an 18. It also is down to individual parents bothering about how notice if their kids are affected or going to be affected by certain things.
Make no mistake though, there are some things in games that are really properly challenging and grown up experiences. Sometimes they can sneak up on you and often come as part of a wider narrative. As an example in Red Dead Redemption, an 18, the vast majority of the game is spent riding around the Wild West on a horse. Yes there are gun fights and shooting. However it is no more or less than watching a John Wayne movie.
“John Wayne – 1961” by 20th Century Fox – eBay. Licensed under Public Domain via Wikimedia Commons.
Cowboy movies sometimes have to deal with complex issues but they come in many flavours. There is always lots of shooting and death but it is the core and the intent of the film, either being light and comical or the more gritty true to life versions that form the spectrum. RDR seemed in the most part to be the former. I grew up watching westerns and we all had six shooters for toys.However about 2/3 through as the character arrives in Mexico a story telling cut scene featured graphic rape scene. Usually you get a sense and feel for the spirit of a game as you play it. You spot that it might be a cert 18 because of the “violence”, this though seemed a shift.
Now lets take Call of Duty (Bear in mind you can get Call of Duty Lego). The single player campaign game in all these games is a full on war film equivalent. However it is more Saving Private Ryan than Where Eagles Dare.
Both great films, but the former is realistic and disturbing, the latter is a playful romp. So single player you will experience some degree of story, probably torture, murder et al. Also some very strong language. It changes a great deal in multiplayer. Here teams of people online shoot at one another and work as teams to capture objectives. The violence is more Where Eagles Dare. It is frenetic and fast paced. The risk here comes from the other people, with headsets and voice turned on then all sorts of bullying and offensive language can take place, equally friendly banter and trash talk. Or, when done right excellent teamwork. The soundtrack itself features very little produced content. The narrative is what you make it.
In many ways the multiplayer COD is less worrying than what can happen on a Minecraft server. Minecraft as a game feature the ability for people to treat one another nicely or nastily. Whilst there is not usually voice you can spend ages building something for someone to come and mess it up accidentally or maliciously. It can also be a great place to learn online etiquette. Minecraft is not an 18. You kill things, but its regarded as cartoon and left alone. It has no story to warrant someone put in some nasty scenes to provide an emotion.
My point is that 18 is a flag, a warning sign. It takes a bit of effort to drill down if you are going to let kids experience the good parts of these.
I am happy for kids to shoot at one another in these games, it is just a twitch reaction game. I am not happy for them to play the single player experiences, those things crafted to shock and engage adults on adult themes. So I wonder if that makes me negligent? Being happy that kids can see some content that is put under an 18 certificate?
This of course is not what is going to happen. These warnings are to remind parents to be parents, to think about what they are doing, to help guide their kids.
Both the predlets are martial artists. They train in punching and kicking in ways that if used properly are very dangerous. They see punches and kicks all the time yet it is all done in a positive way, one that is not about the aggression of the will to dominate others. They are taught to not misuse what they learn in class. Does that mean that playing a fighting game with an rating over their age is not appropriate, often these at 16 like the UFC game or even WWE?
So all this comes down to context. To understanding when game does not mean light hearted play but intense narrative. It means understanding when competition is or isn’t healthy. It requires us as parents to understand a what is going on in these experiences and how we can find the appropriate path for our kids.
The age rating on these things is a relic of a past approach to media. It is very easy to classify 2 hours of celluloid, The 18 certificate is then a barrier to entry to a cinema when a person is selling a ticket, or even buying a DVD.
Games and their associated places are far more complex. They are playing fields where at different times with different groups very different things happen.
I am not totally sure how we can help all the parents out there. Many of course will just not bother to apply any thought at all and just let their kids on anything, on the basis it is easier.