Made as an Intro for the 64k

Introduction in the Intro

Demoscene is about creating cool pieces that work in real time (like "turn in your computer"). They are called demos. Some of them are really small, say 64k or less — these are called Intro. The name comes from the advertising or submission of hacked programs (crack intro). So, the Intro is just a little demo.

I noticed that many interesting works of demoscene, but they have no idea how in reality to do the demo. In this article a brain dump and post-mortem of our fresh Intro Guberniya. I hope it will be interesting to beginners and experienced veterans. The article covers almost all the techniques that are used in the demos, and should give a good idea of how to do them. In this article I'm going to call people by nicknames, because that's what made stage.

Binary for Windows: guberniya_final.zip (61.8 kB) (a bit broken on AMD)

the

Guberniya in a nutshell

This 64k Intro, released on demoparty Revision 2017. Some numbers:

the

the
• C++ and OpenGL, dear imgui for GUI
the
• 62976 bytes binary under Windows, Packed kkrunchy

basically, raymarching (a variant of ray casting approx. lane) the

• group of 6
  the

  one artist :)

the
• made in four months
the
• ~8300 lines of C++ code excluding library and spaces
the
• 4840 lines of GLSL shaders
the
• ~350 commits git

the

Development

Demos are usually released on a demoparty, where viewers watch them and vote for the winner. Edition demoparty gives a good motivation, because you have a hard deadline and passionate audience. In our case it was Revision 2017, a large demoparty, which is traditionally held on Easter weekend. You can look few photos to get an idea about the event.


the Number of commits per week. The big splash — we urgently Hakeem right before the deadline. The last two columns changes for the final version, after demoparty

We started working on demos in early January and released it on Easter in April during the event. Can watch a record of all competitions, if you wish :)

Our team consisted of six people: cce (I), varko, noby, branch, msqrt and goatman.

the

Design influence

The song was ready early, so I tried to draw something based on it. It was clear that we need something big and cinematic, with memorable parts.

The first visual ideas revolved around the wires and their uses. I really like the work of Viktor Antonov, so the first outline is largely copied from Half-Life 2:


the First sketches of the towers of the citadel and ambitious human characters. Full size.


a Conceptual work of Viktor Antonov Half-Life 2: Raising the Bar

The similarities are quite obvious. In landscape scenes, I also tried to convey the mood Eldion Passageway Anthony Chimes.

The landscape created under the inspiration of the this nice video about Iceland as well as "Koyaanisqatsi", I guess. I had big plans for the story depicted on the storyboard:

the storyboard differs from the final version of the Intro. For example, the brutal architecture of the cut. Complete storyboards.

If I would do again, limited to just a couple of photos that set the mood. So less work and more space for imagination. But at least the drawing helped me to organize thoughts.

the

Ship to

The spacecraft was designed by noby. It is a combination of many fractals Mandelbrot, intersecting geometric primitives. The design of the ship was left a little unfinished, but we thought it better not to touch it in the final version.


a spacecraft is a raymarching distance fields, like everything else.

We had another Shader of the ship, which was not included in the Intro. Now I look at the design, it is very cool, and very sorry for him, there was no space.


the Design of the spacecraft from the branch. Full size.

the

Implementation

We started with the code base of our old Intro Pheromone (YouTube). There was a basic crop functionality and a library of standard functions for OpenGL along with a utility file system that Packed files from the data directory into the executable using bin2h.

the

workflow

To compile the project we used Visual Studio 2013, because it don't compile in VS2015. Our replacement for the standard library didn't work very well with an updated compiler and gave a funny error like:


Visual Studio 2015 did not get along with our code base

For some reason we are still stuck on VS2015 as editor and just compile your project using the Toolkit platform v120.


a Big part of my work with the demo looked like this: shaders open in one window, and the final result console results in others. Full size.

We made a simple global keyboard hooks which reset all the shaders, if found CTRL+S:

the

// Listen to CTRL+S.
if (GetAsyncKeyState(VK_CONTROL) &&GetAsyncKeyState('S'))
{
// Wait for a while to let the file system finish write the file.
if (system_get_millis() - last_load > 200) {
Sleep(100);
reloadShaders();
}
last_load = system_get_millis();
}

It worked really cool, and edit shaders in real-time became much more interesting. The interception of file system events and the like.

the

GNU Rocket

For animation productions and we used Ground Control, fork GNU Rocket. Rocket is a program for editing animation curves, it connects to the demo TCP socket. Keyframes are sent on request demos. It is very convenient because you can edit and recompile the demo without closing the editor and losing the position synchronization. For the final version of the keyframes are exported in binary format. However, there are some unfortunate limitations.

the

Tool

Change view with the mouse and keyboard is very convenient to choose the camera angles. Even a simple GUI helps a lot when the little things matter.
Unlike , we had no tool for demos, so I had to create it as you work. Gorgeous library dear imgui makes it easy to add functions as needed.

For example, you need to add a few sliders to control color settings — just enough to make these lines in the render loop (not in a separate GUI code).

the

 imgui::Begin("Postprocessing");
imgui::SliderFloat("Bloom blur", &postproc_bloom_blur_steps, 1, 5);
imgui::SliderFloat("Luminance", &postproc_luminance, 0.0, 1.0, "%.3f", 1.0);
imgui::SliderFloat("Threshold", &postproc_threshold, 0.0, 1.0, "%.3f", 3.0);
imgui::End();

The end result:


These sliders were easy to add.

The camera position can be saved in the file .cpp by pressing F6, so that after the next compilation it will be in the demo. This eliminates the need for a separate data format and the corresponding serialization code, but it can also be quite messy.

the

Make small binaries

The main thing for minimization of binary — to throw out the standard library and compress the compiled binary. As a base for our own implementation of the libraries we used Tiny C Runtime Library from Mike_V.

Compression binaries engaged kkrunchy is a tool made exactly for this purpose. He works with individual executable files, so you can write your demo in C++, Rust, Object Pascal or anything else. To be honest, the size for us was not a particular problem. We kept a lot of binary data like images, so had room to maneuver. Didn't even have to remove comments from a Shader!

the

Floating comma

Floating-point code has caused some headache, by calls to nonexistent functions of the standard library. Most of them managed to resolve by disabling the SSE vectorization by compiler switch /arch:IA32 and removing the call to ftol with the flag /QIfst, which generates code that does not preserve FPU flags to mode truncation. This is not a problem because you can set the mode of truncation of floating point at the beginning of my program using this code from Peter Shafhauser:

the

// set rounding mode to truncate
// from http://www.musicdsp.org/showone.php?id=246
static short control_word;
static short control_word2;

inline void SetFloatingPointRoundingToTruncate()
{
__asm
{
fstcw control_word // store fpu control word
mov dx, word ptr [control_word]
or dx, 0x0C00 // rounding: truncate
mov control_word2, dx
fldcw control_word2 // load modfied control word
}
}

You can read more about these things on benshoof.org.

POW

Call pow still generates a call to the internal function __CIpow, which does not exist. I haven't been able to find her signature, but found the implementation in ntdll.dll of Wine — it became clear that she expects the two registers in double-precision. After that, it became possible to make a wrapper that calls our own implementation of a pow:

the

double __cdecl _CIpow(void) {
// Load the values from registers to local variables.
double b, p;
__asm {
fstp qword ptr p
fstp qword ptr b
}

// Implementation: http://www.mindspring.com/~pfilandr/C/fs_math/fs_math.c
return fs_pow(b, p);
}

If you know the best way to deal with this, please advise.

the

WinAPI

If you can't rely on SDL or something similar, it is necessary to use pure WinAPI for necessary operations on the output window on the screen. If you have problems, here's what might help:

the

the
• Example of creating a WinAPI window
the
• Example of initializing OpenGL, required glext.h and wglext.h
Please note that in the last example, we load the function pointers only for the OpenGL functions that are actually used in business. A good idea may be to automate it. The functions should be called with string identifiers that are stored in the executable file, so fewer functions are loaded, the more space-saving. Option Whole Program Optimization can clean up all the unused string literals, but we are not going to use it for problem with memcpy.

the

rendering Techniques


The rendering is produced mainly by raymarching, and for convenience we used the library hg_sdf. Inigo Quilez (from this point referred to simply iq) wrote a lot about this and many other techniques. If you ever visited ShaderToy, you should be familiar with this.

In addition, we had the results of raycaster — the value of the depth buffer, so we could combine SDF (signed distance field) with the geometry in the confused: b, and apply the effects post-processing.

the

Shading


We applied standard shading in Unreal Engine 4 (here is a great pdf with description) with a drop of GGX. It is not very noticeable, but has a value in the main points. From the beginning we planned to make the same lighting like raymarching and rasterizing forms. The idea was to use deferred rendering and shadow maps, but this absolutely did not happen.


One of the first experiments with overlay shadow maps. Note that both towers and wires cast a shadow on raymarching-land and also correctly intersect. Full size.

Incredibly difficult to properly rendering large areas with shadow maps due to the wildly galloping ratio of screen-to-shadow map-Texel and other problems with accuracy. I also had no desire to start experimenting with cascading shadow maps. Besides, raymarching the same scene from different angles really slow. So we just decided to pay for scrapping the whole system the same lighting. It turned out to be a huge problem later when we tried to match the lighting rasterizing wires and raymarching scene geometry.

the

Area


Raymarching terrain was made numerical noise with analytical derivatives.¹ of Course, the generated derivatives were used for shadow mapping, but also for pitch control rays to accelerate the bypass of the rays of smooth contours as in the examples iq. If you want to learn more, read old article about this technique or play with a cool scene of a tropical forest on ShaderToy. A height map of the landscape became more realistic when implemented msqrt exponentially distributed noise.


the First tests with my own implementation of the numerical noise.


the Implementation of the area from a branch, which decided not to use it. I don't remember why. Full size.

The effect of the landscape is calculated very slowly, because we brutforce shadows and reflections. The use of shadows is a small hack with shadows, in which the size of the penumbra is determined by the shortest distance that was found by traversal of beam of the shadow. They look pretty good in action. We also tried to use the trace method of bisection to accelerate the effect, but it produced too many artifacts. On the other hand, raymarching tricks from Mercury (another demogroup) helped us a little to improve the quality without losing speed.

Rendering of the landscape is improved by iteration fixed-point (left) compared to conventional raymarching (right). Notice the unpleasant artifacts of the ripples in the picture on the right.

Sky generated almost the same techniques as described in behind elevated from the iq, slide 43. A few simple features of the direction vector of the beam. The sun gives quite large values in the HR buffer (above 100), so it also adds some natural color.

the

alley Scene


This view, created under the influence pictures Fun Ho. Our postprocessing effects really helped to create a whole scene, even though the original geometry is quite simple.


Ugly distance field with some recurring fragments. Full size.


Added some fog with exponential change in the distance. Full size.


Wire makes the scene more interesting and realistic. Full size.


In the final distance field added a little noise to give the impression of brick walls. Full size.


In post-processing added color gradient, chroma, chromatic aberration and flare. Full size.

the

Modeling with distance fields


Bombardirovschiki B-52 is a good example of simulation with SDF. They were much easier at the development stage, but we brought them to the final release. From a distance, looks pretty convincing:


Bombers look good in the distance. Full version.

However, it's just a bunch of capsules. Admittedly, it would be easier just to model them in any 3D package, but there was not any suitable tool, so we chose the quicker way. Just for reference, here is the Shader field distance: bomber_sdf.glsl.


But they are actually very simple. Full size.

the

Characters



the First four frames of animation of a goat.

Animated characters — it's just Packed 1-bit bitmap. When playing the frames smoothly from one to another. The material gave a mysterious goatman.


Goatherd with his friends.

the

Post-processing


Effects of post-processing varko wrote. The system is the following:


the
1. to put the shadows from the G-buffer.
the
2. to Calculate the depth of field.
the
3. to Retrieve on the bright side for chroma.
the
4. Run N certain operations Gaussian blur.
the
5. to Calculate the fake lens flare and the glare of the spotlights.
the
6. to Make all along.
the
7. to Make smooth contours with FXAA ().
the
8. Color correction.
the
9. Gamma correction and light grain.
Glare lens owes a lot to technology, described by John Chapman. With them sometimes it was hard work, but the end result delivers.


We tried to use the aesthetically pleasing effect of depth of field. Full size.

The depth of field effect (based on technique DICE) is done in three passes. The first calculates the size of the circle of confusion for each pixel, and the other two passage apply on them two spots from rotating regions. We also make improvement in several iterations (in particular, the blend of numerous Gaussian blur), if necessary. This implementation worked well we and it was fun to play.



depth of field Effect in action. The red picture shows the calculated circle of field to spot DOF.

the

Color correction


In the Rocket there is an animated parameter pp_index, which is used to switch between profiles color correction. Each profile — just different branches of a large fork in the Shader final post-processing:

the

vec3 cl = getFinalColor();

if (u_GradeId == 1) {
cl.gb *= UV.y * 0.7;
cl = pow(cl, vec3(1.1));
} else if (u_GradeId == 2) {
cl.gb *= UV.y * 0.6;
cl.g = 0.0+0.6*smoothstep(-0.05,0.9,cl.g*2.0);
cl = 0.005+pow(cl, vec3(1.2))*1.5;
} /* etc.. */


It is very simple, but works well enough.

the

Physical modelling


In the demo there are two simulated system: the wires and the flock of birds. They also wrote varko.

the

Leads



the Wire add a stage of realism. Full size.

Wires are treated as a series of springs. They are modeling on the GPU using compute shaders. We do the simulation many small steps due to the instability of numerical integration method verlet, which use here. Compute Shader also outputs the geometry of the wires (a number of triangular prisms) in the vertex buffer. Unfortunately, for some reason the simulation doesn't work on AMD.

the

Flock of birds




the Birds give a sense of scale.

Model pack consists of 512 birds, where the first 128 are considered leaders. The leaders are moving on pattern vortex noise, and the rest follow them. I think that in real life birds follow the movements of immediate neighbours, but this simplification looks quite good. Pack renterias as GL_POINTs, which modulated the size to create the impression of flapping wings. I think this rendering technique was also used in Half-Life 2.

the

Music


Usually the music for a 64k Intro done with the help VST plugin: so musicians can use their usual tools for writing music. A classic example of this approach — V2 Synthesizer from farbrausch.

This was a problem. I didn't want to use a finished synth, but from previous failed experiments, I knew that making your own virtual instrument will require a lot of work. Remember how I really liked the mood of the demo element/gesture 61% you did branch with the musical ambient theme prepared in paulstretched. This gave me the idea to do this in 4k or 64k.
the

Paulstretch


Paulstretch is a great tool for a really crazy stretch of music. If you have not heard about it, you definitely should listen to, what he can make out the sound of greeting Windows 98. Its internal algorithms are described this interview with the author, and it is also open source.


Original sound (top) and stretched audio (bottom), created using the Paulstretch effect for Audacity. Also note how the frequency spread spectrum (vertical axis).

Essentially, together with the stretching of the original signal and it still stirs the phase in the frequency space, so that instead of metal artifacts you get an unearthly echo. It requires many Fourier transforms, and the original app uses this library Kiss FFT. I didn't want to depend on external libraries, so in the end we implemented a simple discrete Fourier transform $$ on the GPU. It took a long time to properly implement it, but in the end it was worth it. Implementation of a GLSL Shader is very compact and works quite fast, despite its bradfordbuy nature.

the

tracker Module


It is now possible to wind the coils of ambient buzz, if the initial data is some meaningful sound. So I decided to use the tried-and-tested technology: tracker music. It is largely similar to MIDI², but is packaged in a file along with samples. For example, in the demo kasparov from elitegroup (YouTube) module is used with the additional reverb. If it worked 17 years ago, why not now?

I used gm.dls — the built-in Windows MIDI sound Bank (again the oldest trick in the book) and made the song using MilkyTracker in XM format module. This format was used for many demos under MS-DOS in the 90-ies.


I used MilkyTracker for composing original songs. The final module file from purified samples of tools, and instead set the offset and lengths from gm.dls

The catch with gm.dls, Roland 1996 sound very archaic and of poor quality. But it turned out that there is no problem if you load them in a ton of reverb! Here is an example in which the first playing a short test song, and then a stretched version:



Surprisingly atmophere, agree? So yeah, I made the song, which mimics a Hollywood music and it turned out great. This is generally all that relates to the music.

the

Acknowledgement


Varko thanks for the help in some of the technical details of this article.

the

Additional materials


the
the
• Ferris Logicoma from the group shows its own set of tools to create demos 64k
  the
  the
  • please see Engage, their work at the same competition, which was attended
the
• the Source of some demos Ctrl-Alt-Test
  the
  the
  • code Is 4k and 64k.
  the
  • they also Have an Intro: H-Immersion



---


1. You can calculate analytical derivatives for gradient noise: https://mobile.twitter.com/iquilezles/status/863692824100782080 ↑

2. My first thought was to just use MIDI instead of the tracker module, but it's not like there was a simple way of rendering the songs in the sound buffer Windows. Apparently, somehow this is possible using the DirectMusic API, but I couldn't find how. ↑

Article based on information from habrahabr.ru