GPU based synthesizer
category: code [glöplog]
Hi Guys,
I have been working on some 4k intro, to realize that I’m out of space for 4klang… so I decided to play around with shader based synthesizer.
Currently, the concept is working pretty nice with an exceptional small output size (500-600 bytes for everything). The major problem with this little experiment is me – I have absolute zero tracking skills.
So basically, I need someone help to me with:
1. Provide me with feedback on the concept
2. Test the player on different types of GPUs
3. Create new instruments
4. Create a track for my first intro
I have made a little readme pdf that explains the basic concept.
Also, there is a sample song that I made (keep in mind: zero tracking skills).
You can download the binaries + shader source from here.
Anyway feel like having a little adventure?
BTW, you probably need a proper GPU to play around with this
I have been working on some 4k intro, to realize that I’m out of space for 4klang… so I decided to play around with shader based synthesizer.
Currently, the concept is working pretty nice with an exceptional small output size (500-600 bytes for everything). The major problem with this little experiment is me – I have absolute zero tracking skills.
So basically, I need someone help to me with:
1. Provide me with feedback on the concept
2. Test the player on different types of GPUs
3. Create new instruments
4. Create a track for my first intro
I have made a little readme pdf that explains the basic concept.
Also, there is a sample song that I made (keep in mind: zero tracking skills).
You can download the binaries + shader source from here.
Anyway feel like having a little adventure?
BTW, you probably need a proper GPU to play around with this
Quite an amazing achievement! I think the requirement of GLSL-skills really poses a huge barrier for non-coder-musicians. It sets the level of entry extremely high, leaving only a handfull of coder-musicians to contribute. Not that this is bad if you have one of them help out - it will give you the upper hand in any compo for sure!
Have you considered creating a fixed set of parameters with some sort of generic GLSL template "underneath" for non-coders to play with. I was thinking something in terms of a simple GUI, that hooks into some predefined parameterized GLSL code, which it then modifies as "parameters" changes. Ofcourse, it would limit the possibilities drastically (for those using only the parameter-GUI) but it would definitely allow a lot more people to use it/contribute.
Have you considered creating a fixed set of parameters with some sort of generic GLSL template "underneath" for non-coders to play with. I was thinking something in terms of a simple GUI, that hooks into some predefined parameterized GLSL code, which it then modifies as "parameters" changes. Ofcourse, it would limit the possibilities drastically (for those using only the parameter-GUI) but it would definitely allow a lot more people to use it/contribute.
Punqtured, thanks for the feedback.
I totally agree with you the current implementation set a high barrier and I absolutely agree with you about the need to create a more "structured" user interface for developing the instruments (I was actually thinking of web based).
Anyway, the tool has two concepts that are somehow different to normal composing, the first is the code based instruments and the second is the pattern based tracking.
Right now, I'm kinda more interested in know if the pattern based tracking makes sense? is it workable? and would it work for a real song?
If it doesn't, it drops the entire concept and if it does, it will be really easy to push forward on UI for instruments and effects.
one thing to keep in mind is that creating instruments in 4klang isn't exactly a walk in the part as well.
I totally agree with you the current implementation set a high barrier and I absolutely agree with you about the need to create a more "structured" user interface for developing the instruments (I was actually thinking of web based).
Anyway, the tool has two concepts that are somehow different to normal composing, the first is the code based instruments and the second is the pattern based tracking.
Right now, I'm kinda more interested in know if the pattern based tracking makes sense? is it workable? and would it work for a real song?
If it doesn't, it drops the entire concept and if it does, it will be really easy to push forward on UI for instruments and effects.
one thing to keep in mind is that creating instruments in 4klang isn't exactly a walk in the part as well.
In essense, pattern based tracking is ... well ... standard. As I see it, the non-so-standard thing about your synths, is the way these short sequences are edited and structured. Again, I believe that's actually what most sizerestricted synths end up with after recording/export Renoise XML etc. So that aproach is definitely a viable solution. Somewhere down the road, you could do an actual sequencer, that allows the musician to edit each little sequence and copy/paste, drag/drop every sequence into the full song. This too, may be done using patterns of repetition which most musicians would end up doing in standard trackers/sequencers anyway.
The main thing would probably be to create some kind of layer "above" the synth to "protect" the musician using it. A web based instrument editor would be an excellent example of that. The best possible solution, though, would probably be a standard VSTi plugin, since that would allow any DAW to include your synth. Most musicians doesn't care much about what's going on underneath the nice VSTi GUI. Those who do, can edit stuff directly and achieve even more spectacular instruments.
And I agree on your view on 4klang. It too limits the use of it somewhat, yet it does have a nice VSTi GUI that allows people to load predefined presets, share them and alter them in somewhat understandable ways. It's full potential is probably available only to a very few musicians, yet it does expose paramters that are familiar to musicians that know about synthesizers, hardware or software. So keeping the terms as close to something recognizable would probably be a good idea too, but be aware not to limit the possibilities to that of known synths. Exposing the code directly for the musician to alter, would allow for more interesting instruments, that would otherwise not be possible to create.
The main thing would probably be to create some kind of layer "above" the synth to "protect" the musician using it. A web based instrument editor would be an excellent example of that. The best possible solution, though, would probably be a standard VSTi plugin, since that would allow any DAW to include your synth. Most musicians doesn't care much about what's going on underneath the nice VSTi GUI. Those who do, can edit stuff directly and achieve even more spectacular instruments.
And I agree on your view on 4klang. It too limits the use of it somewhat, yet it does have a nice VSTi GUI that allows people to load predefined presets, share them and alter them in somewhat understandable ways. It's full potential is probably available only to a very few musicians, yet it does expose paramters that are familiar to musicians that know about synthesizers, hardware or software. So keeping the terms as close to something recognizable would probably be a good idea too, but be aware not to limit the possibilities to that of known synths. Exposing the code directly for the musician to alter, would allow for more interesting instruments, that would otherwise not be possible to create.
Just a thought: old-style drum machines show a nice simple way to set up patterns. You have 16 on/off switches to set up your pattern, and you have perhaps 4 patterns.
Change the on/off to a note value, and add an extra layer on top where you can lay out the song with your patterns, and you have a very simple song editor that's at least fairly familiar.
Change the on/off to a note value, and add an extra layer on top where you can lay out the song with your patterns, and you have a very simple song editor that's at least fairly familiar.
psonice, I'm not sure I understand, how is this different that the current implementation?
Do you mean to present and edit of a the patterns notes separately (not as part of the song rows)?
Do you mean to present and edit of a the patterns notes separately (not as part of the song rows)?
Guys, did anyone manage to execute GCPlayer and it worked?
Kudos! The approach sounds viable and the the size is impressive. Even if it goes to 700 - 800 bytes for a more elaborate song it's still ahead of the usual 1.0 - 1.2 kb music budget for 4kb intros. Looking forward to see this unroll.
TLM: I didn't try. Doesn't have the skills nor the hardware required ;)
Oh - and to add to the editing/patternbased discussion. You might want to check out how arpeggiators work. It's basically the same as psonice suggests
Punqtured, oh, now I understand what psonice meant...
It's a nice idea and it definitely simplifies things. One thing I'm not sure about is how you make an arpeggiator that have different length of patterns.
Is there such a thing?
It's a nice idea and it definitely simplifies things. One thing I'm not sure about is how you make an arpeggiator that have different length of patterns.
Is there such a thing?
I think a proper understanding of the very basics of trackers could probably help here. I'll give it a shot :)
A song consists of a number of patterns. Each pattern have a number of tracks and each track has a number of bars/positions
Consider this:
As you can see, in Renoise, all patterns have the same number of tracks (9 in the example) and the tracks in a pattern, all have the same number of bars/positions (16 in the example). Now this is nice, since it probably makes it easier to pack nicely. However, that's only one way to do it. There's several other aproaches, that might work too. Which works best for ease of use and size conservation would have to be tested. I think fixed size is probably easiest/smallest.
Now to continue on the path of arpeggiators, a track in the image above, would be the same as an arpeggiator. Each instrument would have its own "arpeggiator" which can then be repeated at fixed intervals (for example every 64 bars/positions)
Another and slightly more flexible aproach, would be to allow for varying track-lengths. For example, you could do a 4 bar bassdrum track, a 2 bar hihat track and an 8 bar snare drum track. This could then be combined into one pattern of 64 bars/positions like this:
Track 1 repeats every 4 bars and therefore repeats 16 times
Track 2 repeats every 2 bars and therefore repeats 32 times
Track 3 repeats every 8 bars and therefore repeats 8 times
and so on and so forth.
Since you're using Crinkler, you shouldn't worry too much about explicitly allowing the musician to specify repetitions. They pack to practically nothing in the final result anyway.
As psonice suggests, having a fixed number of bars/positions in all patterns (thereby forcing the same number of bars/positions for each track) is probably the best way to start off.
A song consists of a number of patterns. Each pattern have a number of tracks and each track has a number of bars/positions
Consider this:
As you can see, in Renoise, all patterns have the same number of tracks (9 in the example) and the tracks in a pattern, all have the same number of bars/positions (16 in the example). Now this is nice, since it probably makes it easier to pack nicely. However, that's only one way to do it. There's several other aproaches, that might work too. Which works best for ease of use and size conservation would have to be tested. I think fixed size is probably easiest/smallest.
Now to continue on the path of arpeggiators, a track in the image above, would be the same as an arpeggiator. Each instrument would have its own "arpeggiator" which can then be repeated at fixed intervals (for example every 64 bars/positions)
Another and slightly more flexible aproach, would be to allow for varying track-lengths. For example, you could do a 4 bar bassdrum track, a 2 bar hihat track and an 8 bar snare drum track. This could then be combined into one pattern of 64 bars/positions like this:
Track 1 repeats every 4 bars and therefore repeats 16 times
Track 2 repeats every 2 bars and therefore repeats 32 times
Track 3 repeats every 8 bars and therefore repeats 8 times
and so on and so forth.
Since you're using Crinkler, you shouldn't worry too much about explicitly allowing the musician to specify repetitions. They pack to practically nothing in the final result anyway.
As psonice suggests, having a fixed number of bars/positions in all patterns (thereby forcing the same number of bars/positions for each track) is probably the best way to start off.
Hohoho!! This is good shit :D
And here comes Ferris. One of the very select few, that could actually give you that upper hand with a synth like this ;)
Punqtured, thank you for taking the effort :)
I'll explain my approach using your (probably the standard) terminology (mind that in the document I'm using a different terminology).
First, the concept is very similar to your 2nd approach in the way that tracks have a variable length. However, in order for the GPU to be able to process the song structure several things had to be changed.
The differences from what you suggested are:
On the other side of things, you are very right, the concept is different and if I want composers to use it I will have to create a more simplified concept. In a way arpeggiator might do the trick, however, when using this, you give up on many abilities.
Another point to regarding arpeggiator, the entire tracking concept is basically based on a formula, and not even a complicated one, because of that I have had many experiments that sound too repetitive. I have made a lot of effort to escape from that repetitiveness. And it somehow feels that arpeggiator would be a step backwards
Sorry for the super long, technical post...
I'll explain my approach using your (probably the standard) terminology (mind that in the document I'm using a different terminology).
First, the concept is very similar to your 2nd approach in the way that tracks have a variable length. However, in order for the GPU to be able to process the song structure several things had to be changed.
The differences from what you suggested are:
- First, the concept of a pattern does not exists (at all). Each track endlessly repeats it's self. So for example, if you have 2 bar and an 8 bar track, they will "sync" every 8 bars. ie: Code:
T1 [121212121212...] T2 [123456781234...]
- Since there are no patterns, each track always plays the same sequence of notes. let's say that you want to play a different sequence. To do that add another track and switch between them. ie: Code:
T1 [121212mmmmmm...] T2 [mmmmmm343434...] (m = masking of tracks)
- Third change has to do with the bars. Since there are some design limitations of working with the GPU you can not have "spaces" in the track, so let say you have two tracks with both the length of 4, one with 2 notes and the other with 4 notes. ie, something like this:Code:
T1 [1-3-1-3-1-3-...] T2 [123412341234...] ("-" donates an empty space)
In order to have this working in GPU you would need to "change" the length of each bar in T1 to twice it's orignal length. After doing this you end up with:Code:T1 [1 2 1 2 1 2 ...] T2 [123412341234...]
On the other side of things, you are very right, the concept is different and if I want composers to use it I will have to create a more simplified concept. In a way arpeggiator might do the trick, however, when using this, you give up on many abilities.
Another point to regarding arpeggiator, the entire tracking concept is basically based on a formula, and not even a complicated one, because of that I have had many experiments that sound too repetitive. I have made a lot of effort to escape from that repetitiveness. And it somehow feels that arpeggiator would be a step backwards
Sorry for the super long, technical post...
I was actually thinking more of the UI with my simple drum machine comparison.. it's kind of a simplified tracker, which should keep it suitable for 4k, it'd be a lot easier to code as a UI than a tracker, and musicians should be mostly familiar with it. It should just need some note settings (pitch and any parameters you need) adding for each 'switch'.
psonice, can you link an image?
TLM have you thought about something more like this:
- Have blocks of 8 notes (these would be your T1, T2 etc). All the same length and speed for simplicity. Each note just has a pitch. In the synth part, attach pitch to volume, so if pitch is very low (lower than the lowest note) it's zero, otherwise it's 1. (That lets you insert gaps easily).
- Have patterns of length 8, where a pattern is like (T1, T2, T3, T2, T1, T2, T3, T4). This lets you build a more interesting song structure.
- Have perhaps 2-4 patterns play in parallel to give you drums, bass, lead or whatever.
- Have blocks of 8 notes (these would be your T1, T2 etc). All the same length and speed for simplicity. Each note just has a pitch. In the synth part, attach pitch to volume, so if pitch is very low (lower than the lowest note) it's zero, otherwise it's 1. (That lets you insert gaps easily).
- Have patterns of length 8, where a pattern is like (T1, T2, T3, T2, T1, T2, T3, T4). This lets you build a more interesting song structure.
- Have perhaps 2-4 patterns play in parallel to give you drums, bass, lead or whatever.
Drum machine:
That line of squares just above the bottom is what I mean. On a drum machine you just use it to set the drum pattern. Add instrument number (or have 1 instrument for each of those tracks) and pitch, and you have a simple music editor.
That line of squares just above the bottom is what I mean. On a drum machine you just use it to set the drum pattern. Add instrument number (or have 1 instrument for each of those tracks) and pitch, and you have a simple music editor.
That a nice concept... I'll give it a try. Thanks
I've been waiting for a glsl synth since the time I stored .s3m samples in video memory because I was out of real memory. This has a ton of potential and I look forward to the experiments!
@trixter: <grin>
Except pure geekery, is there any advantage having a synthesizer running totally at GPU side ?
Processing power? Saving codespace?
Pure pwnage?