Back-drop for a play...

For the past month I've been working on a project for a friend in my free time (Which seems really scarce these days since I started working at a "real" job again.). In a very small time-frame I chose to make a simple scene-compositing tool to be used as a projected background in a play.

My friend wanted clouds, a lake, some landscape and maybe a bird and a hut, and maybe even a castle (and lots of other things I just said 'no' to.). In the end, time-pressure from work and general life-related issues allowed only the core of our initial plans to be implemented (No bird, no hut, no castle.). Thankfully the cast and crew are talented enough to make up for the short-falls on my part, and ingenious use of shadow-puppetry fills in many blanks (Or so I'm told... I haven't actually seen the play since I have to work in Johannesburg). Unfortunately, the first two shows went on without the landscape, clouds or water being properly implemented... The scene-composition tools were more important, and a deadline from work took priority. People watching the last shows might still see a hut, a bird or even a castle.

Source and instructions here: https://github.com/zskilz/scenes
Actual Demo here: http://zskilz.github.io/scenes/

At its heart, it is really a cut&paste&customise exercise using three.js example code as the source material.
Along the way I used dat.gui and tween.js to do all the tweening of dat.gui pre-sets. I also discovered how simple to use, yet effective localStorage is.

Given how little time I had to spend on this project, I'm happy to have provided something at least usable in the end.

SpaceThreat updated

Just a quick one to let everyone know that the "Wannabe Aliens" experiment has been updated to use the most recent version of KineticJS and the CDN has moved to github.

You'll find the demo page here:

I'm pleased that the minified version of the game is under 20kB... I'm also happy with the new JavaScript workflow provided by RequireJS and Almond. It makes it easy to keep your code-base and deployment-strategy unified-yet-separate.

My previous workflow relied on many custom tools, and by the end, an update to the minified served version required about 10 minutes of tinkering. Now I just need to run the build scripts and push to gh-pages. The process, once set up, takes around 30 seconds. That makes it an order-of-magnitude quicker at deploying changes across all clients. That is the difference between "Urgh... I'll do it just now" and "Lemme do it now quick".

Intro to tonal harmonic theory Part 2: Stuff you have to learn.

Stuff you have to learn

Disclaimer:

This post contains dubious amounts of hand-waving and opinion, and you might get more from it by ignoring the text and just playing with the table below, and the harp from the previous post. For audio to work you need a Web-Audio-API compliant browser like desktop Chrome or Safari.

Introduction

Western music notation is not the de-facto standard in music notation... there are many standards. But western musical terms are the most prevalent, and more often than not, when attempting to collaborate with anyone, you will find yourself needing to communicate using western conventions. So you might as well learn it. It looks hard at first, but you'll get the hang of it!

Next up, a table with the basics of western tonal harmonic theory. In this table you will find the common chords and scales. The discussion part will gloss over why these are the "common" chords and scales. I will sometimes refer back to part 1..

The Table

Choose a dominant note, and then press the buttons to hear the intervals, chords and scales (if you have a Web-Audio-API-compliant browser). The darker rows represent the black notes on a piano, and a "X" in a row means that note will be played. Intervals and chords are played simultaneously, whereas scales will ascend. You will have to use the scroll-bar at the bottom of the table to get to the scales. The source code for the table is here if you are interested.

Discussion

Unlike many tables like this you might have come across, I followed the piano-roll convention used in popular sound production software... where "one row up" translates to "one semitone up".

The "note numbers" column is consistent with the interactive harp in part 1. Choosing a "Dominant note" is analogous to setting the "Base String" in part 1.

This file contains the meat of the table above. It is in essence the first principle starting point for deriving much of western tonal harmonic theory. Hopefully it is enough to get you going.

Western pitch encoding.

This is the naming scheme still in use today to describe the 12 semitones : A, A#, B, C, C#, D, D#, E, F, F#, G, G#.
After G# it "loops" back to A, but an octave higher.

The reason western music theory is filled with oddities in naming conventions is because of historical conservatism. We could have ended up with a pitch encoding which used Arabic numerals instead... in fact, music teachers are forced to do this to get many concepts across (eg. counting out 7 semitones from "B" to show its "5th" is a "sharp"( more on sharps just now). ). The seemingly strange lack of mathematical consistency becomes consistent once you ignore the western conventions and just count out the semitones.

Intervals are just relative spacings between two notes played together. A "5th" will always be some base note and a note 7 semitones above it.

The choice of terms for intervals demonstrate a lack of mathematical proficiency in its originators since the 6th semitone interval is assigned, not a number but, a mystical sounding name instead... "The Tritone". Imagine counting like this in maths class: "One, kinda two, really two, kinda three, really three, four, MAGIC THREE!, five, kinda six, really six, kinda seven, really seven, eight!" (when what you really meant was to count from zero to twelve). People will think you're mad, or worse, an idiot. That is kinda the point to western music theory though... a lot of it is convention rather than sense... which is why, if you wanted to communicate your compositions or play in a band, unfortunately, you just have to learn this... which, once you learn it, you realise that even though it is a bit clumsy (like doing calculations using Roman numerals), it still remains systematically consistent... so at least useful.

Why are only some pitches a "#"

In western music theory they have two operators called "sharp" (♯) and "flat" (♭). Basically a "sharp" is a shift "up" by one semitone. A "flat" is one semitone shift "down".

Back in the day they still thought there were 7 tones (not counting the octave) in an octave (hence the name "octave" from Latin for 8). The tones were not logarithmically equally spaced and looked a lot like the octatonic scale set-up in part 1.

Once it emerged that 12 tones would be a better idea, instead of adopting a 12-toned scheme, they instead adopted the concept of a "semitone" (which you should be familiar with by now). In other words: a semitone is what a tone would have been had western music theory adopted a 12-tone scheme. Instead they kept the 7 tone system and kludged in the missing tones with "semitones". From part 1 it should be clear that the missing notes will make up another pentatonic scale.

That is why only some notes are sharp: the "sharps" make up the "semitones" that were missing from the original octatonic scale. They make up the black notes on a piano.

Scales

The family of Major scales was similar to earlier octatonic scales and they remain prevalent in modern music. The harmonic minor scale is the odd-one-out... it is common for reasons outside the scope of this post... basically, it is around because it has a pleasant sounding system of chords associated with it.

Resonance

I mentioned previously that I would go into this, but I will really only be touching on it. Resonance is the ability of something to sympathetically vibrate with something else which is also vibrating. A String will most strongly resonate with vibrations that are the same as its harmonics. Most strongly with the 1st harmonic, and then progressively less with each harmonic after that. So, looking back to part 1, intuitively, early iterations should produce a set of strings with strong resonances (up to 4 iterations), but then quickly become less resonant (from 5th to 7th iteration), and then progressively more resonant again just before the iterations loop back (the last 4 iterations).

So, using this grossly-over-simplified-hand-waving explanation it seems that chords based on the pentatonic scale will resonate the strongest. Significantly, the Major and minor chords exist in the pentatonic scale and they happen to make up the bulk of popular compositions. Diminished and Augmented chords are basically extensions of the Major and minor into the 12-semitone scheme, and are examples of "dissonant" chords... important in Jazz compositions.

More?

This post has become large enough... If there is enough interest, I might do another post expanding on sets of chords within a scale... but I think you've been sufficiently "introduced" to tonal harmonic theory that I can leave that as an exercise for you, the reader.

Intro to tonal harmonic theory Part 1: Why are there 12 semitones?

Why do we have 12 notes in an octave?

Disclaimer:

This is not a definitive source. Please forgive all the links to wikipedia. I studied physics, electronics and computer science. My understanding of music isn't the same as someone formally trained in, say, western classical music, though I understand enough of it to get by. This represents more an attempt to get across an intuition I've had about the question "Why do we have 12 notes in an octave?". It also acts as a means to test my intuition by actually implementing it. So, I won't bother with historical references, not only because they don't interest me, but since the confirmation to this post is in the code (which is still a work in progress, so I make no claims, etc...) hopefully running in front of you ( if you have the right browser... sorry). I leave the historical referencing and digging to wikipedians and people who are more passionate about that kind of thing. I'm more interested in the types of truths one can confirm through simulation... and music happens to be filled with those...

Introduction

In this post I present an interactive harp using an iterative construction method following a version of the Pythagorean construction method. The construction method will be presented visually rather than mathematically. The source is here if you are interested in the mathematics.

From using the interactive harp it should become clear why there are 12 semi-tones in the modern "chromatic" scale, and some further discussion will show how they became "tempered".

Note though that using some other scheme, or just arbitrarily, you could divide an octave up however you want, with as many notes spaced however you want...

This post specifically deals with the question of the origins of the tempered 12-tone chromatic scale which is most prevalent at any music store or almost standard in any music production software... and to provide an intuitive grasp on the physical origins of the modern tempered 12-tone chromatic scale.

The interactive harp

What follows is a harp made with strings that are "ideal"... meaning strings that might only exist by virtue of mathematics (so no problem for synths). The important attribute of ideal strings is that their harmonics are "perfect"... meaning; the 3rd harmonic is exactly a 3rd of the fundamental wavelength. Real Life strings actually have a 3rd harmonic which is dependant on the mass distribution properties of the string, and is not exactly a 3rd... but more on this later.

Also, these ideal strings are all under equal tension, so only their length need be considered: i.e. Longer strings have a "lower" fundamental (or 1st harmonic), and shorter ones have a "higher" fundamental.

The figure below is playable on browsers that support the Web-Audio-API (Currently only desktop versions of Chrome and Safari).

The figure above starts off as a pretty boring harp. The harp was constructed by taking some base string, and constructing 2 more strings whose fundamentals are the same as the base string's 2nd and 3rd harmonics... which in this case is simply 1/2 and 1/3 of the base length.

The choice of colours for the strings is arbitrary (in this case the hue value is incremented in 12 steps). The string's colour's is mainly a visual means to identify "new" pitches. You will notice that the 1st and 2nd strings are the same colour... this is because 2nd harmonics are so fundamental to music theory that they are assigned the same pitch. The 2nd harmonic also signifies what is known in western music theory as an "octave". Therefore, only the 3rd string will get a new colour... and is therefore the only "new" pitch. Historically, the 2nd harmonic and 3rd harmonic produce the interval referred to as a "5th".

Time to play...

Select a desired western base note with the "Base note" to be the largest string on the harp. Note though that in RL, we can select any base note with any frequency, and the results will look the same. The origins of the western naming scheme is hinted at in the interactive harp, but we'll go more into that in part 2.

What the buttons do:

• The figure can be reset to its initial state by pressing the "Clear" button.
• The "Iterate" button does all the magic... but it is quite easy to follow. Basically, it takes any newly coloured string, and doubles the length of that string until it falls within the initial octave's range (the first two red strings). Using the new long string's length as reference, another string is constructed which is a 3rd of that length. This final string will have a new colour, and will be the input for the next time you press the "iterate" button.
• The "Draw L_n=L₀.2^n/12; n=0..31 ("tempered" scale with 32 notes)" button does what it says... draws the "tempered" scale as reference. The tempered chromatic scale is what you will commonly find with non-sliding modern instruments (fretted guitar, piano, flute, bassoon, etc...). Specifically here I followed the conventions of concert pitch... which means I arbitrarily specified A4 to be 440Hz. There was no other reason for this choice than to make it relevant to western music theory. More on tempering later.
• The "Show Note Values" button displays the western pitch codes: (A, A#, B, C, C#, etc..)
• The "Pentatonic Scale" button resets the harp and iterates 4 times. This produces the well-known and cross-cultural pentatonic scale. The pentatonic scale also contains the musically fundamental major and minor chords, but more on this in part 2.
• The "Octatonic Scale" button resets the harp and iterates 6 times. This produces the well-known family of major scales (i.e. the Major, Natural Minor, Mixolydian and Dorian scales).
• The "Non-tempered 12-tone Scale" button resets the harp and iterates 11 times. This produces the well-known chromatic scale. This specific scale is a sub-set of just intonation tuning. You can press the tempered scale button to compare the difference. More on tempering in the discussion.
• The "Many iterations later" button resets the harp and iterates 52 times. This is to show the limit of this construction method. Any more iterations will over-step the initial chromatic set-up( Non-tempered 12-tone Scale).

Discussion

This version of the Pythagorean construction method can conceivably be performed visually or by ear. Given some basis string, this iterative method will eventually cycle back to somewhere very close to the initial string after 11 iterations (not counting the set-up as an iteration); dividing an octave into 12 (almost logarithmically equal) semitones. Repeated iterations will repeat this cycle every 12 iterations after that until it becomes a bit of a mess... So, in short, the reason there are 12 semitones, is because of the 3rd harmonic... A iterative construction based on the 3rd harmonic will produce 12 distinct bands which appear to "loop" back every 12 iterations. This construction will produce a pattern consistent with the circle of 5ths ( remember, the 3rd harmonic produces a interval on the harp historically referred to as a 5th (More on intervals in part 2))..

Small mathematical detour and tempering...

In Real Life, a string's 3rd harmonic is dependent on many factors, and generally isn't "perfect". Generally the factors string manufacturers take into consideration are tension, length, diameter and mass (length, diameter and mass = mass distribution.). So, is there some different 3rd harmonic that would make this iterative process cause the 11th iteration to loop exactly back to the beginning? The answer is in the definition of the semitone.

The semitone:
An octave is a doubling or halving of length or frequency... so, an "octave" isn't a length, or anything physical. It is a ratio. What we've been doing here is dividing lengths by 2 or 3, and sometimes multiplying by 2. So, the semitone is therefore some value, which if we multiplied some original value by that value 12 times, it would produce exactly 2 times the original value.
So, lets say x is the magic semitone value, then x*x*x*x*x*x*x*x*x*x*x*x*L = x¹²L = 2L,
or simply, x¹² = 2.
Therefore the semitone ratio, x = 2^1/12.

In 12-toned set-up, the 3rd harmonic would occupy the 19th string from the base string. For 12 equal semitones to exist, instead of the upper 5th being the base note times 3, it should rather be the base note multiplied by our semitone ratio, x, 19 times. Which is x¹⁹, or 2^19/12, which on your calculator would give 2.99661415 (rounded to 8 digits). Notice how that number is damn near 3.

Now, since you can't get a string with a 3rd harmonic which is "perfect", this at least gives one a physically attainable target. By tailoring mass distribution properties on strings and getting close to this target, manufacturers are "tempering" the string to fit into the 12-tone tempered chromatic scale, resulting in superior harmonising. Historical experimentation into the tempered scale often only considered the tuning of the instrument, but modern manufacturing takes into account the physical manifestation of harmonising (tempered string vs. tempered scale), which in essence is resonance. But more on that in part 2.

Guitar scales interactive.

Hi there.
While I'm making my way back to HTML5 game dev, I thought I'd share some work I did on musical theory.
This is a small interactive HTML5 Canvas guitar scales generator written in JavaScript. For standard guitar set-up, the Tuning Base Note should be set to "E" and the tuning to "Standard".
The scales wrap around, so the pattern doesn't often start on the BaseNote/Key/Root. It basically shows you all notes on the guitar that exist within the chosen scale.
As a exercise, guitarists familiar with bar-chords will can try find the three major chords, the three minor chords and the one diminished chord existing within the E major scale.

Guitar scales

BaseNote/Key/Root:

Scale:

Handedness: right left

Tuning:

Tuning Base Note:

© Copyright by Petrus J. Pretorius. All rights reserved.

Game-play concept for Rogue-Like project.

EDIT(7 July 2013): Code moved to github: https://github.com/zskilz/manband
github.io page not up yet, sorry, been busy. To play download the zip and open manband.html in Chrome or FireFox (Requires webGL).

Ok, so making a fully fledged Rogue-like in three weeks was a bit unrealistic... but considering I didn't know three.js when I started, I think I did ok, though unfortunately I'm one day late on delivering something playable (on Chrome at least).

Here is what is missing, or buggy, from the game-play experience:

• EDIT: Firefox, weapon sprites not showing... 
• There is a bug where the  bites from small zombies are ineffectual - I know why, and I will fix later.
• Inventory management. 
• The ammo indicator needs a reload display to let you know when you can shoot.
• Improved passive AI to help avoid structures.
• Flocking, for group behaviour.
• Group commands with the right click.
• A proper map with multiple arenas.
• Game events - for dialogue, death-screens, etc.
• Help screen with legend.

The graphics are very debug, but I make no excuses for this... The focus is still game-play. I will only focus on polish once the game-play is to my satisfaction. 

So there you have it. Deadlines are silly, and silly people like me always tend to over-estimate their abilities. To complete the above list I now guess would take maybe another week... but then again it could be two, or three!

Quite a bit of work went into this project so far, and if you cruise the code you'll notice that many parts are still in development and left dangling uselessly, hoping to be used soon. So, much of what you don't see, has some place-holder code waiting to be implemented... but then again lots of code is just plain not there yet.

Next up: As a priority, I will do some bug fixes, and add any improvements to the inventory system since currently the weapon with the highest damage points gets automatically selected, which means that you need to reload and try avoid some pickups to try others. Also, a reset button is desperately needed.

After the important fixes, I suspect the pace of development will slow down since I need to spend more time on polishing my personal website, and to focus on getting possible financing so I can hopefully make games for a living...

Anyway, go check out the game-play demo here

That link again: https://github.com/zskilz/manband

Game challenge 2 update.

TL;DR: My grand Rogue-like game idea is turning into a zombie shooter... NOOOOooooooo!

I've bitten off more than I can chew and as of today, I broke the combat system, the loot system is in a primitive state, and the team artificial intelligence is, well... let's just say you can't call it intelligent.

I'm busy fixing the combat system to be more projectile friendly... If I don't get it to work I can fall back to the melee-only system whose implementation I'm not happy with. I aim to have a playable demo up on monday, but it will be the equivalent of the 1st week of the 1st project in terms of being presentable. The focus will be on core game-play concepts.

The following sacrifices will have to be made to the great deity of Keeping-timely-dead-lines:

• Player vs. world only - the AI is still too dumb. 
• Loot will be kept in it's primitive fps-pickup style of functioning... and since it's single-player, no point in rushing the team loot management.
• Since the AI is dumb, you going to have zombies, because zombies are dumb... sorry.
• If any other characters appear, they'll be about as bright as the zombies.

There is currently a nice fluidity to the game-play I hope players will appreciate; in spite of the lack of inspired  graphical representation.

Something completely different

TL;DR: The new project is coming along... one week left!

So, I'm getting along with my Rogue-like project which is currently consuming most of my time. Things are looking better but the game-play isn't near the point where a public presentation is wise. The initial set of characters will be tiny and you can expect about four types of weapons and three types of armour as loot. Coins might make an appearance, but only as upkeep for your army, acting as a pacing mechanism to add urgency. The upkeep rate will also be proportional to difficulty level.

I have settled on microphysics.js to temporarily provide dynamics. I will need more time to properly research physics library options for html5 development. The options are varied and solutions that use CPU or OpenCL acceleration would be preferable.

Anyway, there's the work-in-progress screen-shot and I apologise because it isn't very pretty :

The brave adventurer surrounded by dogs.

Progress on Rogue-like with working title "manband":

• Adapted terrain example to generate the arenas using Perlin noise. (I'd like to spend more time in a future post on this...)
• An arena can be defined with a small set of parameters, enabling random generation or defining tile-sets in a JSON file.
• Basic character class with RPG-related stats.

What's next?

• microphysics.js sphere/AABB limitation limits structure placement to axis-aligned orientation only. 
• Terrain dynamics will have to be faked - abstracting dynamics to spheres-on-a-plane and setting the sprite heights directly from the terrain's data.
• Character dynamics and passive AI.
• Loot management.

I plan to use dat.gui for tweaking character parameters during testing and for the loot management system.

Till next time...

Bump in page-views.

TL;DR: Some people came to look at my blog, and now I'm blogging about it...  yeah.

My blog received a welcome boost in views when kineticjs referenced my first personal game challenge which used their library... Thank you kineticjs.

What this whole experience has opened my eyes to is the power of market-specific references. Being referenced on a site that deals with specialist information attracts a considerably larger audience than posting about it on google+ or facebook if you're not known in your field.

Meanwhile, in unrelated news...

Graphics test.

TL;DR: Canvas character sprite experiment here.

Since I haven't set up my server to properly serve the compiled JavaScript as static files yet, I'm not getting any 304 codes when serving my JavaScript files. This is a problem since the above demo relies on three.js, which takes up about 200kb (webGL+Extras)... Quite a bit to load, and I should really get the compiled files in the right place on my server so they can be loaded statically, and give a 304 if a cached version is found on the browser. This will mean only the first page load will be slow. Consequent loads will be much quicker.

It's also for this reason that I would not dare include the script references in this blog, since it would just slow down the loading of the blog way too much.

Anyway, go check out my experiment.... It shows what any Rogue-like needs... lots of character(s)! Ha ha, ha... ha.

A warning to other programmers writing graphics tests: The problem with writing graphics tests is that they can become quite hypnotic, and if you're not careful you can spend almost as much time staring at it like a mesmerized fool as you do actually implementing it...

New game challenge...

TL;DR: I'm starting a 3d web game Rogue-like project. OK, no pictures, you can go away now. :)

Next stop, 3d and more complete game dynamics. The previous release was a retro-inspired title, so it is fitting that we continue along that vein, but this time on a more serious note. This project will tip its hat at all the Rogue-likes out there...

The great thing about Rogue-likes is that you can still get away with using ASCII characters, and by providing a text description, you can convince a player that a "@" character is a human. I've had the experience of playing Dwarf Fortress for 2 hours where I would imagine a dog instead of the "d" character actually running around. Reading a book has a similar effect. Your eyes sees words, but you imagine worlds.

But I'm not just doing it for the snob-appeal, it also allows me to develop a game-play experiment without having to expend massive amounts of energy on modelling lots of items and characters, but rather focus on the game-play and game-dynamics... and attempt to do so in under 3 weeks.  It is also for this reason that I believe concept art would be pointless for this post. ;)

Project brief (Man-band (working title)):

• Rogue-like 
• ASCII character sprites to represent creatures
• Simplified physics using box2d-js
• 3d environment, though all character interaction and AI will happen on the horizontal 2d plane
• no enter-able buildings, all structures effectively block or spawn creatures.
• Game-Play:
• Real-time (possible turn based option, but not a priority)
• all action is top-down 
• aswd/arrow keys to move. (Relative and absolute options)
• mouse to aim and attack. (full-screen, but no need for pointer-lock API)
• troop commands:
• assign to/select squad
• move to (move to selected spot -behaviour affected by hold/free)
• hold/free (stick to formation or chase enemies for a certain radius)
• rejoin
• chain commands.
• Game-Dynamics:
• Simplistic revenge plot with minimal dialogue and lots of fighting and looting.
• Simplified statistics and items (no magic/acid/flame/etc... just physical damage)
• Arenas represent basic location blocks. The player moves his troops from arena to arena to progress.
• Creature generator structures in arenas will cause creatures to be spawned. Creatures will also periodically spawn in neighbouring arenas. You need to destroy  generators to stop the spawning and gain loot.
• Game progress involves clearing arenas of enemy creatures, saving people and recruiting followers and arming them with loot. When you have enough stats, weapons and followers you can take on the boss arena, upon whose clearing the game completes.

Agreeably not a unique idea, but I hope to implement it so the game-play will be satisfying despite the lack of "real" graphics.

For now, implementation details and plot details will be held back until I have more to show.

Javascript hosting site up.

TL;DR: I have a new site up which will serve my game scripts.

The site is hosted here: http://zskilz.webfactional.com
The first demo is available there: http://zskilz.webfactional.com/demo/1

Also, the "Game challenge complete!" post now relies on properly importing JavaScript from this site, rather than having to paste JavaScript into the blog post. This means I now have one place where I can apply my fixes. Fixes will be reflected in the games in my posts without any extra editing of the blog.

The site, though not impressive looking yet, gives me the ability to define JavaScript bundles : usually a collection of scripts written for a game I've finished working on. The bundles get concatenated and served as one big file which can be included in a script tag.

This provides me with a more flexible deployment strategy for free games. The site will also serve as a portal people can possibly donate to or purchase from in future.

My next focus will be on a 3D mini-game, to be announced on Sunday.

Pointer Lock API means more input for your mouse.

TL;DR: FPS mouse input soon to be natively supported in Chrome browser.

Scheduled for release with Chrome 18, comes the much over-due, [plugin/Java-applet/NaCl]-free, ability to make games for the web with FPS style (or custom) mouse input.

Firefox has an event intercept hack to allow control of the mouse pointer. Up till now, that has been the only plugin-free way to provide FPS style mouse input for a web app. Cross-browser solutions required a Java-applet or additional plugin to "capture" your mouse input. Hardly ideal solutions. A new W3C draft proposal, the Pointer Lock API [1], hopes to standardise this issue.

 The Pointer Lock API is for mouse input what what input.focus() is for keyboard input... It "Locks" your mouse pointer in place, limiting pointer movement to an area, ensuring the pointer input stays relevant and doesn't go off the edge into another window. Mouse movement values (not just co-ords, but the mouse deltas also) will be included in the mousemove event. This combined with hiding the cursor is what is required to code FPS style input (amongst many input schemes) for your app.

Since this feature is ripe for abuse, the specification demands that the user is first asked if they want to lock their mouse. The lock can be released by pressing the "Esc" key. The "Esc" key event would have to be promoted to a top-level browser event whose default action cannot be prevented.

The discussion about abuse is moot I feel. The HTML JavaScript "window" object allows you to resize and re-position a browser window arbitrarily. So whatever abuse a mouse lock feature would enable has already been enabled by the window API - remember "Rick Rolling"? (a video played a particularly annoying Rick Astley hit and any attempt to close it was prevented by the script simply moving the browser window away from your franticly flailing mouse pointer). The fact is sites that chose to exploit that feature in an abusive manner only got visited at most once. It might be good for a gag, but you can only fool them once. Also, the Pointer Lock API at least has the "Esc" hatch, making it safer than the existing window API anyway. And since the pointer can't go outside the locked area means the script can't go clicking on other windows.

Have a  look at the html5Rocks article for a more technical overview [2]

Links:
[1] http://dvcs.w3.org/hg/pointerlock/raw-file/default/index.html
[2] http://updates.html5rocks.com/2012/02/Pointer-Lock-API-Brings-FPS-Games-to-the-Browser 

Game challenge complete!

TL;DR: I made a derivative "game". W00t!

Hello. Below you will find the completed game challenge. CLICK ON IT TO START THE GAME. Use left and right to move, and up to shoot. Press "Esc" or click anywhere outside the game to pause. Click the "RESTART" button to reset the game.

EDIT(2013/05/13):The Audio has been fixed. Code moved to my github account. Fire-Fox doesn't appear to automatically cache the tentacle drawing so I will still need to optimise that to increase frame-rate in Firefox. Chrome frame-rate is good.

So, I managed to achieve my goals in the allotted time, and I even managed to get it to run in a blogger page.
You might notice that the prototype post is gone. This is because I had to remove it. It wasn't written to play well with other game scripts on the same browser page (i.e. the default blog view when people visit.). It hijacked your arrow keys for the whole page, which is not acceptable web-app behaviour. This version behaves better, and pauses/mutes when you go away from it and only captures your keys after you click on it. It would be a good template for me to use to build future browser based games.*

Many features one might expect from a proper game are clearly lacking in this game. This mini-game does not provide a full-game experience, but rather is aimed as an interactive replacement for a web picture, since the whole game-code is just under 50kb non-obfuscated (not counting the libraries, of which jquery will be cached on most people's browsers, and kinetic.js which is about 40kb). All geometry and audio resources are created on-the-fly, i.e. no pictures or audio get loaded, they get temporarily generated on the client machine. It represents an experiment used to learn some new technologies, and is only the first in many (hopefully).

If you care to know some technical details and credits:
I used kinetic.js library for canvas management and jquery.js library for event stuff. I ended up just using the vanilla Web Audio API implementation for Chrome build 10.0.912.77 and it gave me no hassles. In fact, I had such a blast with it I hope all browsers will have native implementations of the Web Audio API soon! If you don't have Chrome, you should just not hear any audio (sorry).

Thanks to html5canvastutorials.com.
Thanks for your interest, and please leave comments if you have questions or suggestions.

Battle of the audio APIs

TL;DR: Native Web Audio API is better if you're making games.

Which is better? Google's Web Audio API [1] or Mozilla's Audio Data API [2][3]?

 Well, actually that is not a question which can be easily answered since the two, though sharing some use-cases, are quite different. Also, to emphasise the difference, Mozilla's Audio Data API specification, officially since 15 December 2011, falls under a more general "MediaStream Processing" API specification [3]. Just browsing the two specifications will show you that they are quite different animals and that the Web Audio API specification is currently more mature (less likely to change).

To really get at the core difference between the two APIs, known that Google's Web Audio API can be,(and has been [4]) implemented in JavaScript using Mozilla's Audio Data API. So, immediately it is apparent that the Mozilla API is a lot more "low-level". That is, it gives you some core functionality which allow access to HTML "audio" elements' internal data. The specification, now more generalized for "MediaStream Processing", also allows for access to HTML "video" elements' internal data.

So, the specification championed by Robert O'Callahan and Mozilla is great for managing media assets, and performing streaming, extracting, injecting or mixing operations on raw data from audio and video elements. Microphone input would be considered a media element under this specification. The big use-case will be for in-browser video-conferencing or web-casting applications.

What is the Web Audio API then? It allows the user to build complex audio graphs, place sound sources spatially (including features like Doppler effects and HRTF ), trigger sound sources in a timely fashion, and comes with a set of pre-defined filters and effect generators. So, everything you need to make a soft-synth or a computer game's audio.

Now, since you can implement Chris Rodger's Web Audio API specification on top of the Mozilla specification in JavaScript, why have native support for it? Answer: Performance.

A JavaScript implementation of the Web Audio API, though functional, will run much slower than a native implementation. When making a game, you need processing time for game logic. You don't want CPU cycles tied up performing a set of computationally expensive convolution operations in JavaScript. Also, since the API is only a specification, a good implementation can hand off processing to hardware accelerated back-ends (e.g. on a Creative EAX card), or a browser maintained audio thread to help boost performance. For a complex scene with many filtered and spatial sound sources playing simultaneously, like in a game, you'll need all the performance you can get to keep the user experience responsive.

 Also, since the audio sources in the Web Audio API are easy to fill with custom buffers, there is little reason why both specifications can't live side by side. AudioContext's MediaElementAudioSourceNode method seems like a move in the direction of inter-operability between the two specifications. The  MediaStream  API can manage audio streams between peer-connected clients while the Web Audio API "renders" the received data in interesting ways.

So, in conclusion, for me, there is no battle, though I personally prefer using Web Audio API since it is more geared toward the game use-case. I hope that both specifications are refined to be more complementary and get accepted and endorsed by W3C [5].

 Links:

• [1] http://www.w3.org/TR/webaudio 
• [2] http://wiki.mozilla.org/Audio_Data_API 
• [3] http://www.w3.org/TR/streamproc 
• [4] https://github.com/corbanbrook/audionode.js
• [5] http://www.w3.org/TR/audioproc/

Game rush

TL;DR: I'm starting on a simple html5 game project.

I've decided to make a series of mini-games aimed at html5 compliant browsers. Each game will have a personal goal in familiarising myself with some key html5 technology or support library (like three.js). 

Ok, so here goes. The very first game concept "art"  is for a very derivative game called "Invaders from space! OMG!". It's a working title...

You control the TOSPOT (Tank Of Space Protection from Otherworldly Threats, or whatever other rude sounding anachronism I can think of before I make the game), your planet's last line of defence against the unknown dangers from outer space. The antagonists, being beings from outer space, are clearly more technologically advanced than you and their nimble frames can effortlessly evade your every attack. On their slow, meandering (yet evasive) path down to the planet surface, they shout admonitions and insult your inferior technology. The twist is that the invaders from space, although rude, are peaceful and just want to give you a group hug.

The end.

The personal aim of this project is to familiarise myself with the web audio API defined for html5 and to brush up on the html5 canvas element. This game will be 2d and I'm setting a deadline of 2 weeks for implementation. Let the challenge begin, tomorrow! (I mean, it's Sunday... you seriously expect me to start working on this on a Sunday?)

Change of plans... again.

TL;DR: I'm giving up trying to make an engine for now since better free tools exist already, and they're in better state than I expected. Go look at these pretty demos from other people, or go listen to these demos made by other people, if your browser supports it.

Maybe you spend all your time working on something everyone else seems to be doing, but no one quite got it right, or maybe there exists a solution, but it is out of your price range... Sometimes you could be wasting your time doing something someone else (meaning better connected groups of individuals and corporations) already did, quite a while ago, but you just didn't delve deep enough into the research to find it. And by "you" I mean me, cos that's what I kinda did. Oops. Also, I'm running out of time, and I want to be known for games, and not engines.

I was working hard on creating modular wrappers for audio sources, and most recently managed to add a spacializer class for attaching audio sources to rigid bodies. Being single minded about making a game engine I never bothered to read the W3 specification for HTML5's audioContext object which includes everything I was trying to achieve in terms of audio routing in my own project [1]. Searching for examples of implementations using Chrome as browser convinced me that I was wasting my time [2] [3] [4]. The web audio API is mature enough in Chrome to do everything I set out to do, so I might as well use it and hope that future adopters of HTML5 will provide a half decent implementation of the autioContext specification.

Further down the line, I expect that the audioContext will make its way into QtWebKit. When this happens I will gladly base the next iteration of the engine on QtWebKit to create a self-contained executable.

In my defence, time has moved on since I started the project early last year... The state of HTML5 was matured, but not complete. It appeared to have a bright future, but that future wasn't "now" back then. I don't think we're at the "now" stage yet, but it does seem close with M$'s adoption of HTML5 as an alternative application framework in windows 8.

I had a plan to develop a game engine with emphasis on a robust scripting environment. At the time I was trying to get away from web dev for just a bit. I decided on Qt for its cross-platform Win/ Linux and MacOs support. Somewhere after the Bullet Physics integration I started to address the issue of GUI. I followed the over-painting example from QT to use QPainter for any GUI text. Even later, after doing the OpenGL script interface I wondered if I could not save a lot of effort to just use QtWebKit for the GUI, since I was already well versed in CSS and HTML... it seemed natural. Round about this point I came back to see the state of HTML5 and WebGL, and I decided that for now, they are good enough for my purposes, limited by time as I am. Seems I went full circle to land back at web-dev.

Sigh.

Anyhoo, the video below shows the point my engine got to before I decided to give up making engines.. It demos the audio spacializer somewhat (STK sources get attached to rigid bodies and the mixer parameters get updated from the motion-state callback). I was about to add the delta-Position variable to the bodies' motion-state for Doppler effects, so that is not done. (Also no texture support... the colours are generated by a simple shader):

I plan to open the source for this project soon, I just want to first remove the direct inclusion of STK in the source and link to the library instead. There is some documentation, but it is mostly undocumented. All in all, though mostly a waste of time, it was a great learning experience. Don't hold your breath for the source though... priorities.

I will keep you updated on the progress of my HTML5/WebGL game development adventure.

Links:
[1] http://dvcs.w3.org/hg/audio/raw-file/tip/webaudio/specification.html#Spatialization-section
[2] http://www.html5rocks.com/en/tutorials/webaudio/intro/
[3] http://www.chromeexperiments.com/webgl
[4] http://chromium.googlecode.com/svn/trunk/samples/audio/index.html

Almost done with audio

TL;DR: Still no pretty pictures. Sorry.

Since last post:

• Mixer.  Per-source volume and pan.  It can have multiple modules as sources, and forces any mono sources to fill all output channels (dependent on pan setting). 
• Stk Voicer support. Can essentially make polyphonic versions of any Instrmnt and trigger it from the script environment.
• Started on Stk Filters, but apart form delay, most of them seem ill-fitted for my purposes (see below).

Currently only stereo linear-cross-fade panning is supported by the mixer module. I'll re-visit multi-channel placement later. Audio positioning for game objects will only rely on volume and pan for now.

What's still left to do?

• Ignore all Stk Generators for now... they are too low-level to merit script instantiation.. They might make a comeback as LFO-type control inputs in the future...
• Ignore most Stk filters. Filters in Stk seem to be meant as helper classes for internal workings in custom Instrmnt derived classes. Many filters do not normalize and without extreme tweaking aren't useful in an intuitive real-time modular fashion. Compromise: Script instantiated filter which can be set up as a HP,BP,LP or BS filter. Internally an IIR Stk filter can be used with a known calculated set of normalized coefficients for Butterworth filters. This will be sufficient for games.
• Ignore score files for now... The effort which will have to go into creating a scoring environment is beyond the scope of a gaming project. It is a love of mine, hence the long interlope. I will come back to this, but not yet. 
• Ogg file loading/playback for music needs to happen (Easier to compose music in Renoise for now).
• Stk Effects to support : Chorus, JC-Rev, Echo ... the rest can come later.
• Implement tying motion-states to audio sources.

Tick these off and I'll have enough audio tech to my liking. 

After that? Menu GUI and exposing the keyboard-map to the script environment, then we'll be close to ThingMaker version zero point something!

Ok, so it is still a way to go, but an end is in sight.

No pretty pictures here, please move along...

TL;DR: I made a sound from within the scripting environment, and it was hard!

Disclaimer: programmers only.

It has been almost a month since the last post and no "visible" progress has been made...
Lots of code have been committed for the purpose of making a couple of extra lines of script work.

Basically, I had to write an interface class for an abstract "sound module"... which is something with an internal audio buffer and references to inputs and outputs to other sound modules. Script interfaces had to be written for connecting modules together with logic to detect any feedback between modules. The only sub-classed objects so far is a master module, which abstracts the "default" audio output device, and a Stk Instrmnt wrapper class.

When I designed for using Stk, I planned around the Stk base interface classes as much as possible, and being obsessed with the sound module idea I settled for the most common audio processing virtual function shared by all higher level Stk interface classes... namely : StkFrames &tick(StkFrames&frames).

Since this most common tick function processes a block of audio frames, I thought it best to have the algorithm which processes the audio frames in as large a chunk as "real-time" permits... Stk defaults to 512 frames, which at 44k1Hz equates to about 11ms (good enough for games!).

Wasting of time, feeling stupid and doing more work than necessary happened when I decided to design for feedback support after already having "chunk" optimizations in place.

What am I talking about? Say you have an instrument which you plug into a mixer. That mixer goes through an EQ and into a splitter. That splitter then has an output to a chorus effect and another to a re-verb effect. The chorus goes to the master output, but the re-verb goes back to the mixer the instrument is plugged into. Why do you want to do this? Well, why not? It might sound cool... who knows, but the point is you might just NEED it somewhere.

The problem is now apparent since it is clear that the mixer's second input is dependent on the output of some later stage which is in turn dependent on the mixer's output. The best you can hope for in a digital system is to allow the mixer to do a single frame's tick, propagate that tick though all modules until you update the module the mixer input is dependent upon which will then serve as the input to the mixer's next single-frame tick operation.

So, if like me, you went ahead optimizing for chunks, and then wanted to go and have a special feed-back case which just does single ticks, then you're probably also half-wasting your time (I'm sure it's possible, just probably not with the way I went about it.).

What I kept from this misadventure was just the ability to detect if there is feedback when a connection is created, and throw a script exception if so, not allowing the connection. A lot can be done still with just feed-forward modular audio networks, so apart from the wasted time, I'm not too bumbed.

So, I now have the master module which is the entry point for connection to other modules and the Stk Instrmnt wrapper  (which currently supports all the sub-classed instruments Stk has to offer). Further sub-classed modules still need to be added to complete the whole sound experience, but the minimals are coming together nicely.

The basic functioning of the audio engine is like:

• audio callback requests buffer
• audio engine processes the master module
• master module processes its source modules recursively

...still not noticing any non-stupidity-induced breaking sound artefacts, so all is still well and real-time(ish).

Hopefully the next update will have some video with "actual" sound. In the distant future, I hope to re-visit the modular feedback issue. 

What went wrong this time? (Adventures with STK)

Sorry, no pretty pictures in this post... Also, this post is moderately technical in nature, so browse away now if you are not interested in audio-programming jargon. You'll be happier for it. If you know Qt and STK well, you might find this post slightly amusing. If you are new to Qt and STK, you might not know what I'm talking about at all. This post is personal, I guess.

I've made the first steps for physics, interaction and graphics, and since I'm interested in making desktop games, it is only natural to do audio at some point...

I thought it would be easy to integrate Qt's audio functionality when using the qmake tool-chain... add qtmultimedia to the config in your .pro file, and include a QAudioOutput class somewhere and trigger sounds from that... Right?

Well, it kinda is... but not really... QAudioOutput is a reasonably low-level interface class and can read from a custom QIODevice derived class (pull mode, good for streaming... limited real-time application) or you are provided with a QIODevice which you can write to once a notify interval has elapsed (push mode... more real-time).  A QIODevice "talks" in raw bytes... so either way you still need to take care of formatting your audio data correctly to match the stream parameters (code for this can be seen in the Generator::generateData method in the Audio Output Qt example ). Also, no mixing/routing functionality is provided... you're expected to implement that or use some third party library like STK. So, although the QAudioOutput class provides a solid cross-platform starting-point, it still requires you to implement a lot of the low-level, broiler-plate, byte-array, endian-conscious code. Also, this part of the Qt framework is under active development so it might drastically change  in the main Qt distribution as projects become merged into it. It is inevitable that technology will pass you by... but it is all part of the learning process.

For a while I've wanted to use Perry Cook & Garry Scavone's STK (https://ccrma.stanford.edu/software/stk/) to provide the basis for a soft-synth audio framework for my game engine. As a preference I decided on RtAudio and STK over PortAudio and CSound, though I know much is interchangeable and CSound has a great deal more features. I was drawn in bySTK's well balanced compromise between latency (#1 consideration for games), footprint and readability.

The aim was to have an audio engine class which gets instantiated along with the script environment, which would also provide script interfaces into a range of STK classes and to dynamically connect them in a modular fashion... Since STK can be compiled without RtAudio dependence I though it best to stick with the QtMultimedia module since I was relying heavily on Qt already.

One can conceivably tie STK's non-real-time build in with a push mode implementation involving QtAudioOutput, but, after wasting much time, in the end experimentation with RtAudio  made me decide to use it instead of QtAudioOutput. The biggest factor for me was that RtAudio already deals with floating point samples and transparently does the conversion to raw audio for a wide range of audio formats. Also, it has a relaxed open source license and is cross-platform (Linux/Mac/Win). My DSP courses dealt mostly with audio signals represented as numbers in the range -1.0 to 1.0, so RtAudio is a more intuitive fit in this regard also.

On the plus side, choosing RtAudio over QtMultimedia also provided the additional ASIO support in windows ( although ASIO support is compiled and tested, I'm  defaulting to dSound on Win32 platforms for now) and RtMidi cross platform MIDI IO support ( which has also been compiled in but is waiting untapped). The STK source files have been statically incorporated into my project not because I planed it, but because I was initially only using some elements of STK... Eventually all of STK source got included. I might eventually move back to just linking to the STK library. Right now I don't care much because the code footprint is small and all the STK classes are pre-compilable, so they don't affect consequent compile times much (exception is lesson 1).

Lessons learned:

1. Complacency can make you forget the classics: Changed a define in your project file? CLEAN! REBUILD! (And don't forget the cleaning part. Too embarrassed to say how much time I wasted dealing with puzzling bugs after forgetting to do this).
2. RtAudio's closeStream() method should never be called from a destructor of a class your RtAudio instance is part of... safer to have a pointer to a RtAudio instance which you manually delete after step 3.
3. You need the "bool done" tick-data element you see in all the STK code examples... no escaping it if using the tick() callback functionality. In the main tick() callback check for data.done and return -1 if it is true; The audio engine class destructor should set the "data.done" value to true and should sleep for a couple of microseconds to allow any pending callbacks to return a -1... only then is it safe to delete the RtAudio instance without a hang (on my system at least).
4. Building STK with Qt(Qt Creator/qmake) MINGW: If like me, you added all the stk/include and stk/src files to your Qt project, you'll need to add some defines and library dependencies to your .pro file to get everything linking and compiling right. To get the compiler setting and project defines for STK, run STK's configure script from within a msys shell to generate a makefile. Reading this makefile will tell you what defines and libraries you need to add to your Qt .pro file (you can also look at the VC project files that come with STK but it is more verbose and the linker syntax is different). 

So, what do I have?

Not much really, just getting everything compiled together and first steps of simple integration for now. Tones are generating; instruments are playing; filters are filtering; effects are effecting even... but only C++ tests hacked into the audio engine's initAudio() method for the interim. The initial tests show promise. No audio script interfaces or script definable modular flow graphs have been implemented yet... that is next.

I still have a lot to do with the sound-programming before my next update... sorry, I know I can ramble sometimes. This post went on for longer than intended.

Useful links:

• http://www.mingw.org/ (msys doesn't come with Qt Creator installer)
• https://ccrma.stanford.edu/software/stk/ (STK Home page)
• http://www.music.mcgill.ca/~gary/rtaudio/ (RtAudio Home)
• http://web.mit.edu/carrien/Public/speechlab/marc_code/ADAPT_VC/rtaudio/doc/html/ (MIT RtAudio Tutorial)
• http://www.music.mcgill.ca/~gary/rtmidi/ (RtMidi Home)