This week Dave Taylor/Davee Commerce and Robin Winter had a special on Treet.tv about lots of the virtual world projects in Second Life that Imperial College London have been up to. It is a great show to watch to see the variety of ways Dave has got Second Life working from public information, targeted patient experiments and doctor training.

The doctor training and evaluation that appears around about 32 mins in Dave says. “This is where we have our virtual patients, and these patients are controlled by software actually outside of Second Life. That software has a knowledge of the patients physiology and condition.” He also explains there are 3 wards and 3 patients in each giving 9 levels of difficulty in scenario.
“We are using this to research how we can asses trainee doctors at different levels of training”. “We have tested about 60 doctors so far on this”.
I am glad this is out in the public as this has been part of the work I have been doing in SL. I can’t explain exactly what does what as its a private project but as Dave points out the patients and the interactions are controlled from outside of Second Life, my part in SL is the broker talking to that external model. I also ended up building the dynamic menus and handlers in world. The menu’s are based on the data coming back, and align to the correct place in world so they are designer friendly. This was built before the web on a prim existed, and we aimed to do everything in world. As you know handling text can be a problem in SL and variants of Fasttext and xy text came to rescue. Though rezzing a dynamic button and making it know what it is supposed to do is a non trivial task. This was also before HTTP in world servers were stable so SL is the controller asking the external software what to do next.
It has been a fascinating project, as has its follow on ones that have increased in complexity and in interactions. Making SL a component in a system not the sole piece of the project makes for a greater richness and flexibility. After all SL is not a database/data handling application.
What is great is that Robin, who is one of SL’s foremost designers (along with his other half) and has been for years(he built the original Dublin sim), is able to craft animations and objects and then trigger them into existence using our message protocol, after the external software model tells my broker code that its got some changes to display.
There are a few of us pushing the bondaries of data interchange with SL and also with opensim and other virtual worlds. I hope this helps people understand that we can do very complex integrated tasks using the best of a Virtual World and the best of a traditional server application. Integration is the key.

I was looking around for a quick way to use the wonderful Reactivision camera based marker tracking. Ideally I really wanted a fully working Reactable, but without a projector and the music and light software and the physical elements like a glass table I cant really do what I want to do in the time I need to do it.

I bumped into a project from a few years ago the tangiblaptop that was aimed at using the laptop screen rather than a projector to display the things that happen to the markers that are also placed on that surface but picked up by the camera. It looked good but I thought I needed something with a bit more variety.
Then I saw this Uniducial . A small library and and couple of scripts to drop into Unity3d.
Within seconds I had objects appearing and disappearing based on the markers they could see in the camera view.
(I have done a few things with these before as did Roo back in the day
However I think the unity3d gadget is going to be very useful indeed

I was just replying to a note asking about Opensim and Freeswitch based on the fact I have it working(ish) on Ubuntu on a cloud server. I have been meaning to share where I am up to, though I have not found a complete solution to the problem I am having. However this snippet may help someone, or they may be able to help me

My Opensim and Freewswitch are running Ubuntu Karmic on a cloud service. Both opensim and freeswitch get built rather than binary distributions. However the ini files and config should be the same (as thats the beauty of this).

Freeswitch works by really just opening a conference call which any avatar will dial into. I am not sure how much digging you have done into the problem but there are a few things to look out for.
I am assuming you have followed the main instructions for config etc http://opensimulator.org/wiki/Freeswitch_Module

I had this all running from 6.8 onwards but recently moved to 0.7 I noticed in bin/config-include that there was now a line in StandaloneCommon.ini for Freeswitch not just the Opensim.ini. I think I had to uncomment that when I moved to 0.7

The main thing is to make sure that Freeswitch is started first and ready , then spark up opensim.

The default for the region in Opensim is to not have voice enabled at a land parcel and region level. In Hippo I enter god mode and set both the estate options and the parcel options to allow voice. Having done that (only needed if reloading the terrain or something major, I usually logoff and log on again with the view and it gets enabled. That one has caught em a few times.

The other thing to consider is some of the viewers, in particular the more recent open source ones are not allowed to package the SLVoice components with them. That one caught be when I asked someone to use Hippo on windows. The Mac hippo is old enough to still have the voice DLL but the newer windows one did not have it! SL Viewer 1 should of course be OK, but I think it is still not advisable to use SL Viewer 2 on opensim. Imprudence discusses a patch http://imprudenceviewer.org/wiki/How_to_Re-enable_Voice_Chat which is how I found out the root of the problem

The server firewall is another thing to check, as is the client firewall. As the ports are potentially different as freeswitch is a completely different application to opensim and the viewer can get blocked.

Should all of the above be ok then its console time for both Opensim and Freeswitch. There are some spurious errors it seems as things try to establish connections. However if you get a couple of clients connected (assuming you get the voice enabled in teh client) you can go into freeswitch and use the sofia commands. sofia status and sofia status profile internal both give a bit of information. I am no expert on the commands but I have been able to see if clients have connected from opensim.

This is where we get to a problem I now have. Freeswitch seems to not be working for everyone who comes to my sim. In my testing I used a mac and a windows machine on my own network at home but both talking to the remote cloud server. That has always worked (though technically it shows as the same IP address twice). I patched in someone else across the country who was using SL viewer 1 and we had a conversation, so I thought it was all working. However a few other people, when we have tried a larger meeting, have experienced problems.

It has been a mix of using a viewer with no SLVoice in it, firewalls but also some strange timing behaviour. It is quite difficult to test but that is why I have been using the sofia commands.

The first person to connect will generally get hold music playing. The second person to connect will enter the call, there is a beep as that happens the hold music ends. However you cannot always hear anyone speaking.
I suspect this is the client firewall operating. i.e. Opensim will talk to freeswitch at a server to server level to patch the person in. the Sofia status tends to show me all the users patched in to the freeswitch console. After that its the SL clients doing the work. When I get no voice response it tends to break the call for everyone. I have not found out how to check that or tell freeswitch to ignore bad calls. Getting the once that break it to logoff so there is only 1 person (i.e. me) in the call sometimes takes a few minutes before the hold music kicks in again. There are some sofia reset commands for the profile that I have dabbled with.

It was only a few weeks ago I hit this snag, the people using my freeswitch are not always in a position to mess with their company firewalls so it has been hard to get a test rig that fails consistently to try and debug it.

I have been meaning to write this down somewhere to help others but wanted to try and fix the problem first, but not many people are using or trying freeswitch.

Having said that if you get the voice connected (probably the god mode parcel audio solution at the start of this) you may find it just works. It could be my server having some bandwidth or memory issues etc. Its a tricky one to spot.

Good design of any sort is really effortless, or joyous for those on the receiving end of it. Something that has had a whole load of great design go into it is the new gaming classic Red Dead Revolver. Non gamers and “serious” types may just consider this as trivial as hula-hoop. A toy that a few grown ups enjoy playing with. Well…. it is. At one level of abstraction it is a football, a hoop and stick. On another level, it, and the current generation of well crafted gaming experiences are a fantastic example of good design and talent.
Red Dead Redemption is a cowboy gaming experience. It is an exceptionally large free roaming area interspersed with a plot that takes you from set piece to set piece. You can if you want just go for a ride and see what happens though.
The first thing that most people latch onto, with good reason, is the graphics and the animation in the game.

Given where we were only a few years ago the graphics quality, the detail on things like the horse animations, the size and scale of the terrain, the flora and fauna and even tumbleweeds is very good. Still pictures do not do it justice. Xbox 360 or PS3 though it just works. Just think for a moment the amount of graphic design time that has to go into both the size and scale and the intricate detail. The flowers on the plants, the mane on the horse. Even the bullets in your bandolier are all created by someone. So as a graphic design task, even with tool and middleware support this is a monumental undertaking. Of course tiling, and cut and paste comes into play, but just consider the person hours of skilled design tool usage then add to that the overlaying of the design of “where” all these things go and can go.
Under the covers there is of course code. Programming and detail in allowing things to happen, chaining effects together, determining where and when a bullet has hit. As a programmer I know that most people do not see the code under any of this, but it takes as much design effort and talent as the visuals. The system architecture and middleware combinations becoming the “where” all these things go.
However I think that many people in most enterprise businesses and alike will understand a little about IT (from using it all the time) and maybe a little about visuals from having to create the odd powerpoint. Clearly not the same but at least in the general area. People probably have a moderate understanding of testing (though not the mind numbing repetition and test case coverage that goes into knowing something is right)
The things that people don’t have to get involved with and that have really evolved so much and been taken seriously in the production values of high end games are things around the sounds and voices that you hear.

The sound design is generally so well done, creates so much atmosphere and is in some senses more transient than the visuals that it almost melts away. Also the acting quality is just miles beyond the early fumbling attempts to read badly written dialogue we used to see and hear. When the characters in the game talk to one another in cut scenes or as part of atmospherics in a town it feels real. Of course the dialogue has to be layered with western style occasionally over the top elements, but films that we passively sit and watch have all sorts of over the top characters.
Threading all this together is the script, both a story arc and then the micro stories that form at key points in the game progression. Thrown in are also random events that happen around the place as you travel around, thefts and challenges which you choose to engage with.
Its all highly immersive, and very entertaining. That is without even engaging in what is in effect a completely different use for all these assets, the multiplayer games. These take place in the whole environment, fellow real people being cowboys and travelling the land living their own stories.
Now clearly I am a gamer, but I do not always feel compelled to complete a storyline. However recent months have seen Modern Warfare 2, Uncharted 2, Heavy Rain and Red Dead Redemption all making me want to complete the storyline and not ending up disappointed. There are lots of games but these stick in my mind for being good stories and for me wanting to, and actually bothering to complete them. Many people will of course not spend 40+ hours on a single gaming experience of this sort, though they will of course spend 40+ hours grinding on Farmville, or breaking jewels in Bejeweled and alike, or maybe watching Eastenders or Coronation Street repeat soap plots. It is though all good as far as I can see.
Games and gaming experiences, both ones we create for ourselves and ones we are directed through are as memorable as any traditional film or TV experience. The effort and design going into them warrants the time and attention to explore them. For me it is of course business and pleasure, research and a release, which makes it doubly valuable.
Well done Rockstar games (again)

One of the most amazing developments over the past couple of years has been the explosion in creativity tools that are available to anyone and everyone, combined with the ability to share creations with others over the net. The creations can be businesses, presentations etc, funny things, appeals for help or art. Amongst all that there is also games. I grew up in a time when there were arcade cabinets that we drooled over and pumped 10 pence pieces into, we were then treated to the home computer boom. We were given the ZX81/C64/Spectrum, later the Amiga as tools that let us , should we wish to learn our craft create all sorts of things. We did not really have a distribution network other than word of mouth and posting disks around. Something happened to the homebrew market as the PC rose to power. Things got complicated, programming kits and licenses got expensive so we managed to lose an awful lot of homebrew to what became a massively monolithic games industry.
Now there is a shift again. The creativity tools are back and on the previously closed and expensive to licence too platforms that took over.
If you are a gamer or a content creator, a Second Lifer, a designer or any remotely interested in computing and animation or an engineer or teacher you really need to try the original Little Big Planet creator tools on PS3. The whole point of Little Big Planet is create.play.share They provided a palette of objects, rich 3d clip art if you like, combined with mechanical programming logic (motors, pulleys, switches, rods) which let you create all sorts of visually rich experiences with very very simple tools. Or you can just play platform levels with friends and have fun too. All those levels can be shared online. It has been a stunning success. As with all user generated content platforms the depth of human creative talent tends to astound the tool makers.
Now we have Little Big Planet 2 on the way and as a natural evolution of the tools we are going to see some fantastic creations in that. Take a look

The ability to combine things into an experience for others using gaming elements is not restricted to the PS3. The tiny little Nintendo DS (tiny only size not sales) has Warioware DIY. This is an extension of the minigame ideas of Warioware. You are presented with a quirky 5 second task with no real explanation and you figure it out. The graphics are often like something Terry Gilliam would do or are very cartoony, but the games work. Quick ideas executed well. The DIY game is really a collection of games but also the tools to create them yourself. A visual programming environment enabling you to create triggers and win condition combinations. For a programmer is may be annoying to go through the dialog pieces to get to the tools but for people who are not programmers it really starts to make you one. A small graphics and animation package and a music sequencer are also in the game/package. I have not tried it yet but apparently there is a Wii download that lets you get to and share/play you DS creations. That’s next to try.
The important thing here is an evolutionary path for talent to emerge from anywhere, for people to be able to find out if they are good at creating game ideas, combing graphics and sounds and having fun.
LBP2 and LBP approach it as aside to the basic game. Dropping you into a sandbox to play and helping you create amazing things right away. Giving a taste for creation and innovation not just consuming the levels thrown at you.
Warioware DIY makes you work a bit more, its a “go on then create a mini game then if you can” a bit of help but really a dressed up development kit.
UGC virtual worlds and places like Second Life or the more programmer extensible OpenSim also fit into this sort of homebrew model. Its a creation tool as much as a consumption tool with the added layer of events and people online thrown in.
These then can lead people who are interested and talented to tools like Unity3d. With that you are on a PC/Mac. You have all the tools available to you to write proper code, develop proper games from scratch. If you find you are a 3d graphics person there are lots of creation tools from free to very expensive, if you find you can do music or textures the tools exist likewise. Unity3d as a development environment lets you or a team work together to create things. The things created then will just work pretty much anywhere. The simplest being on a stand alone file on a website. Which of course means you have massive distribution potential.
As in a previous post the next large step of creating massive shared online experiences gets a little more complicated, but with people being able to do the things they can now with LBP2/DIY/SL/Unity3d in an out of the box type way is already amazing.
With a few tools, lots of middleware helping it is possible to create very engaging experiences and interesting art on any platform. The 4 way needs of programming, graphics, audio and story/game mechanics meet in various ways on all the creation platforms. The key though is that anyone, and I mean anyone! can have a dabble in any or all of them and find an outlet and talent they did not know they had, or fulfil their potential.
What are you waiting for go and make something somewhere.

The past few weeks I have had my head in an installation of Drupal 6.16 the content management system that sits very nicely on the LAMP stack. So I got to spark up another Ubuntu linux server off in a cloud somewhere,using the great apt packager on ubuntu/linux to go and fetch the extras I needed AND their dependencies. All the extra instructions I needed were on slicehosts forums for various pieces of config. So a full OS, all the extras I needed without having to trawl trough patches and dependency trees manually. In the good old days the stuff was pretty disorganised. If you came to it all fresh, or had been away for a while the “obviously you would have x, y or z” would be quite a pain.

The whole apache, mysql, php installation and config is also very straightforward. Yes you still need to know a few of the more arcane system command lines, or be able to look them up and editing even with nano or vi on a terminal is still rather annoying (terminals being a hangover from before we had more complex machines on the front end of servers). However, doing what would be the simplest task in a drag and drop world in a command line typing all the paths correctly etc makes you feel you are in charge of the whole thing. Which is why sysops are usually quite stroppy
Drupal’s install goes like a dream to. Again a whole set of extra modules are contributed to the let you do other things with your content, thinks like the CCK (content creation kit) which layers some new fields to be able to add to the page creation forms you make.
Back in 1998 I wrote content management systems, mainly in Lotus Domino, I know the problems and also the sort of things you need to do in the systems. So for me Drupal was great, once you get used to the naming convention of the template overrides and the ability to use views (SQL selects) on the data its all pretty slick. It is of course a long while since 2008, and it is interesting that whilst there were lots of commercial CMS packages that attempted to emerge none of them seem a patch on Drupal and even on WordPress. That is for most things, most web applications it is a pretty good fit.
Of course with any themeable template based system with a multitude of user contributed modules and gadgets there are going to be times when things just are not where you need them to be for your particular layout or information design. I spent a fair bit of time with one piece of data and layout trying to do it “properly” in the end I just changed the module, which let me put the class id’s in that I needed to make more sense of the display for the CSS. Not ideal but the point was it was there to do.
Commercial software has all to often been put in place to keep you away from the engine. Opensource be it opensim, drupal, wordpress, linux, freeswitch etc really do let you be the mechanic on the engine if you want, but you clearly don’t have to as the slickness of the design ethics in these applications through crowdsourced cooperation is quite stunning.
To do the full thing from commissioning a server to creating data structures in Drupal and then adjusting templates and style sheets is still a great swathe of skills needed, but when you have been in this for so long you know the patterns and roughly how things need to work. The ability to look up and search for problems, similar situations and generally fix on the fly though really helps to and can’t be understated. Much better than routing through a cupboard of manuals as we had to back in 1990!
Gotta love the web

This is not a rant about the constant patching of my PS3 everytime I turn the thing on, nor about the minor updates hear and there in various operating systems and platforms. Well actually it might be, but its not intended to be rant at all.
Whilst playing the wonderful Just Cause 2 on the Xbox I bumped into a few bugs and glitches. They all happened around the large radio antenna dish up in the mountains.
The first saw me accidentally get inside of the buildings. The buildings tend to be texture walls to climb so once you are inside they seem to be invisible as there is not texture to render. This was amusing until I realized I could not get out at all. I was not able to spoof an evac and had no ammo to self harm so I had to quit Still these things happen.
I then popped back to the same place and found a little glitch with the top edge of a building.

i.e. I am standing on nothing due to some odd collision detection.
For me I do not regard these are terrible flaws but it does raise the question of how, with such large expansive experiences it is ever possible to get enough test coverage and QA in place. Just Cause 2 has 400 sq miles of terrain, trees, building water. Vehicles moving around. Clearly the physics engine can be tested with a relative few vectors but when the place is built by hand, building placed its almost impossible to not have a few errors in place. Equally with a free roaming game its hard to tell when something is a bug.
I had driven a speed boat straight at a yacht. The speed boat was forced underwater with me still driving it, I had a submarine in effect. That is less of a bug a more an unintended consequence, the thing was it then became my mission to try and recreate it, which I eventually did.
Testing and coverage is actually not an instinctive thing to do as I have found with some clients. Where an atomic action happens with the same start and end conditions, but where it is held in a different part of a flow or sequence each time it is not always so necessary to run a test for every permutation and combination. Test the changes, test the extremes and test the critical path.
The software I used to build way back for internal corporate systems testing was a straightforward job in many respects, as we started to get more and more service, more and more permutations and many more functions it clearly has got a lot harder.
It is why I am not surprised that some tests really don’t get done in favour of patch later over the web, though at the same time I suspect some don’t get done because “hey we can patch it later”.
Anyway I am not moaning about Just Cause 2 as I am still in awe of its size and scale, just as I am with GTA IV and will no doubt be with Red Dead Redemption. Free roaming FTW!

In my exploration of what unity3d does and how easy it can be to get things going I was about to set upon implementing the Craig Reynolds flocking code in the environment. I have done a fair few things related to using this flocking algorithm. Not all were published publicly, though I did about 4 years back create an organizational structure viewer where on selecting a person their related people would flock around them so that you could navigate through a organization based on relationships and drag flocks around.
The principle of the flocking code is very simple. Each individual in a flock has a motivation to aim for a destination, but to back from a collision with any other member of the flock. The code used is pretty much the same everywhere based on Craig originals. Flocking can provide some dynamic insight into relationships between objects, in particular when more than one flock, or the focus of the flock can be changed.
Before I set about my coding example I had a look around, and not surprisingly I found that the Unify community wiki had already done the work needed in both C# and JS.

Once set in motion the boids (as they are referred to everywhere) a cast out into the world. They then try and reach a point at the centre of the flock, never bumping into one another, as they get close they back off and then try again. An element of random direction provides an interesting display. Bear in mind these are simply acting under a simple rule, the path and shape of the flock is just emergent from those simple rules.

I have simply published a version of that on this page (it will do the plugin download for you) as you need to see the beauty in both the complexity and simplicity of movement.
As you can see a still just does not do it justice.

So now I have this I can see what I can apply it to. (Flickr Flock of Photo maybe? )