Bug 309

Summary: Improve key group handling (off_by) with a flexible approach to polyphony
Product: LinuxSampler Reporter: Jacek Roszkowski <j.roszk>
Component: otherAssignee: Christian Schoenebeck <cuse>
Status: RESOLVED LATER    
Severity: enhancement CC: naumovvladislav
Priority: P5    
Version: SVN Trunk   
Hardware: All   
OS: All   
Attachments: The patch
The patch v2.1
Corrected again
Patch v2.2

Description Jacek Roszkowski 2019-03-09 07:15:08 CET
Created attachment 94 [details]
The patch

Hi.

This is a work in progress to enhance linuxsampler's key group handling with polyphony. The patch is focused on sfz engine.

Key groups in sfz are configured by two separate opcodes: group and off_by/offby.
http://www.sfzformat.com/index.php?title=Group
The format itself is ambiguous, because there's a <group> section and group opcode. Setting the same group opcode value for two regions (one <region> in sfz stands for one sfz::Voice instance in linuxsampler) doesn't mean exclusive playback. Regions will play simultaneously until they are muted with another region with group value equal to off_by value of a playing voice. This will shut down all voices regardless of polyphony settings.

All <regions> with the same group value should have the same off_by value (ARIA engine reports discrepancy if it's not like that).

Accodring to http://www.sfzformat.com/legacy/ default group and off_by value is 0, which would make instruments exclusive by default. To overcome this issue ARIA engine sets off_by to MAX (4294967295), so by default regions belong to group=0 and are muted with regions in group=4294967295 (but there are no regions in this group by default). Contrary to gig, group=0 and off_by=0 are valid settings for sfz.

There's a very interesting tutorial which shows the potential of these two opcodes to program accurate hi-hat muting behaviour.
http://www.sfzformat.com/index.php?title=Cymbal_muting

polyphony opcode
http://www.sfzformat.com/index.php?title=Polyphony
It limits a number of concurrent <regions> with the same group opcode value. In my opinion when the limit is reached the engine should release the oldest voice first, and it should be released regardless of note_selfmask setting.

All <regions> with the same group value should have the same polyphony value. Default value is undocumented, it's set to MAX in ARIA engine, with polyphony=0 it prevents any <region> from being played within the same group.

Gig regions (in current trunk) with the same KeyGroup value (for example 1) behave like sfz regions with group=1, polyphony=1 and off_by=1. Gig SelfMask setting is being ignored in current trunk

note_polyphony opcode
http://www.sfzformat.com/index.php?title=Note_polyphony
It limits a number of concurrent <regions> with the same group opcode value triggered by the same MIDI key. In my opinion when the limit is reached the engine should release the oldest voice first (that's not how ARIA works), but regarding note_selfmask setting. The idea of self-masking is explained below.

All <regions> with the same group and key values should have the same note_polyphony value. Default value is undocumented, it's set to 0 in ARIA engine, which means there is no limit for note_polyphony. Maybe it should be unified in linuxsampler with polyphony default setting.

note_selfmask opcode
http://www.sfzformat.com/index.php?title=Note_selfmask
Self-masking makes instruments with long sustain more natural, because quieter voices will not mute louder voices (unless polyphony opcode limit is reached or a <region> with group opcode value equal to off_by opcode value of a playing voice is triggered)

This concept is present in a gig format with a SelfMask field inside a DimensionRegion class. The patch tried to implement support for gig engine self-masking with default values of NotePolyphony=1 and GroupPolyphony=1, but now I see it's flawed and the lower velocity voice will be shut down anyway by relaseByGroup flag. I will submit a correction later.

All <regions> with the same group and key values should have the same note_selfmask value. Default is on.
Comment 1 Christian Schoenebeck 2019-03-09 12:49:48 CET
Thanks Jacek for your patch!

@Devs: Postponed for now due to several issues:
https://sourceforge.net/p/linuxsampler/mailman/message/36607166/
Comment 2 Christian Schoenebeck 2019-03-10 16:13:16 CET
*** Bug 300 has been marked as a duplicate of this bug. ***
Comment 3 Jacek Roszkowski 2019-03-12 16:42:27 CET
Created attachment 96 [details]
The patch v2.1

I've changed Voice interface to use two separate functions (GetKeyGroup and GetOffBy) to be more clear.
I've changed the internal algorithm to correctly (I hope) support self-masking for gig engine.
Comment 4 Jacek Roszkowski 2019-03-12 19:14:46 CET
Created attachment 97 [details]
Corrected again

Corrected again.
Comment 5 Jacek Roszkowski 2019-03-12 19:19:13 CET
Created attachment 98 [details]
Patch v2.2

Correction of a correction ;)
Comment 6 Christian Schoenebeck 2019-04-23 19:46:07 CEST
(In reply to Jacek Roszkowski from comment #0)
> Created attachment 94 [details]
> The patch
> 
> Hi.
> 
> This is a work in progress to enhance linuxsampler's key group handling with
> polyphony. The patch is focused on sfz engine.
> 
> Key groups in sfz are configured by two separate opcodes: group and
> off_by/offby.
> http://www.sfzformat.com/index.php?title=Group
> The format itself is ambiguous, because there's a <group> section and group
> opcode. Setting the same group opcode value for two regions (one <region> in
> sfz stands for one sfz::Voice instance in linuxsampler) doesn't mean

Actually I think an SFZ <region> is the exact same thing as a gig region, sf2 region, etc. : simply a continuous span of keys on the keyboard. A voice on the other hand is one instance of a active playback object within the engine implementation, each having its own filter instance, envelope instance, and so on. Each note usually spawns one or more voices.

> exclusive playback. Regions will play simultaneously until they are muted
> with another region with group value equal to off_by value of a playing
> voice. This will shut down all voices regardless of polyphony settings.

Is there any practical usage of using the group opcode without off_by and the other opcodes mentioned by you?

> All <regions> with the same group value should have the same off_by value
> (ARIA engine reports discrepancy if it's not like that).

I see now that SFZ allows a much more fine graded control compared to the exclusive group concept of gig.

> polyphony opcode
> http://www.sfzformat.com/index.php?title=Polyphony
> It limits a number of concurrent <regions> with the same group opcode value.
> In my opinion when the limit is reached the engine should release the oldest
> voice first, and it should be released regardless of note_selfmask setting.

Well, on doubt we can add our own opcode to control the voice stealing algorithm to be used for that. But I agree that by default an old voice should be picked of course.

> All <regions> with the same group value should have the same polyphony
> value. Default value is undocumented, it's set to MAX in ARIA engine, with
> polyphony=0 it prevents any <region> from being played within the same group.

If polyphony opcode is used more than once for the same group AND with different polyphony value, then I would just print a warning. IMO that's simply a user error and I don't mind which one of the polyphony values would be picked in the end.

> Gig regions (in current trunk) with the same KeyGroup value (for example 1)
> behave like sfz regions with group=1, polyphony=1 and off_by=1.

Would off_by=1 actually be required in this example? My expectation would be that the behaviour of:

group=1, polyphony=1 and off_by=1

to be identical with:

group=1, polyphony=1

> Gig SelfMask setting is being ignored in current trunk

That's correct. That was still on my TODO list for near future.

> All <regions> with the same group and key values should have the same
> note_polyphony value. Default value is undocumented, it's set to 0 in ARIA
> engine, which means there is no limit for note_polyphony. Maybe it should be
> unified in linuxsampler with polyphony default setting.

Why should note_polyphony value be identical within the same group? For me it makes absolutely sense to handle note_polyphony separate from the other opcodes controlling aut release of voices.

I could also imagine the note_polyphony be allowed on group level, as well as region level at the same time. Then the note_polyphony's value on region level would override the "default" value on group level.
Comment 7 Jacek Roszkowski 2019-04-23 21:42:16 CEST
(In reply to Christian Schoenebeck from comment #6)

> Actually I think an SFZ <region> is the exact same thing as a gig region,
> sf2 region, etc. : simply a continuous span of keys on the keyboard. A voice
> on the other hand is one instance of a active playback object within the
> engine implementation, each having its own filter instance, envelope
> instance, and so on. Each note usually spawns one or more voices.

You're right, my explanation was too brief.

> Is there any practical usage of using the group opcode without off_by and
> the other opcodes mentioned by you?

There are two approaches to muting (choking) cymbal samples:
1. Using region with "special" MIDI note not used anywhere else, which doesn't spawn any voice (*silence) and has proper off_by value in sfz.
2. Polyphonic aftertouch message (not supported in linuxsampler).

> > Gig regions (in current trunk) with the same KeyGroup value (for example 1)
> > behave like sfz regions with group=1, polyphony=1 and off_by=1.
> 
> Would off_by=1 actually be required in this example? My expectation would be
> that the behaviour of:
> 
> group=1, polyphony=1 and off_by=1
> 
> to be identical with:
> 
> group=1, polyphony=1

Yeah, right, actually gig regions wouldn't act like that anyway, because of the condition itEvent->Param.Note.Key != HostKey() in gig::Voice::ProcessGroupEvent().

> Why should note_polyphony value be identical within the same group? For me
> it makes absolutely sense to handle note_polyphony separate from the other
> opcodes controlling aut release of voices.

As far as I can remember sforzando VST (ARIA engine) reported discrepancy of different note_polyphony values in the same group. I treated it as a reference without considering other variants.
Comment 8 Christian Schoenebeck 2019-04-24 19:59:18 CEST
(In reply to Jacek Roszkowski from comment #7)
> > Is there any practical usage of using the group opcode without off_by and
> > the other opcodes mentioned by you?
> 
> There are two approaches to muting (choking) cymbal samples:
> 1. Using region with "special" MIDI note not used anywhere else, which
> doesn't spawn any voice (*silence) and has proper off_by value in sfz.
> 2. Polyphonic aftertouch message (not supported in linuxsampler).

Yeah, I got that, but just using group without anything else does not do anything, right?

> > > Gig regions (in current trunk) with the same KeyGroup value (for example 1)
> > > behave like sfz regions with group=1, polyphony=1 and off_by=1.
> > 
> > Would off_by=1 actually be required in this example? My expectation would be
> > that the behaviour of:
> > 
> > group=1, polyphony=1 and off_by=1
> > 
> > to be identical with:
> > 
> > group=1, polyphony=1
> 
> Yeah, right, actually gig regions wouldn't act like that anyway, because of
> the condition itEvent->Param.Note.Key != HostKey() in
> gig::Voice::ProcessGroupEvent().

That's also clear, my questions are actually targeted solely on current expected SFZ file format behaviour details, to avoid that I am missing something. Because I am not familiar with all details of the individual SFZ format implementations / interpretations out there.

> > Why should note_polyphony value be identical within the same group? For me
> > it makes absolutely sense to handle note_polyphony separate from the other
> > opcodes controlling aut release of voices.
> 
> As far as I can remember sforzando VST (ARIA engine) reported discrepancy of
> different note_polyphony values in the same group. I treated it as a
> reference without considering other variants.

Mja, I would just ignore their behaviour on that. Unless you can think of a good reason why note_polyphony should be identical among all same groups, then I would simply not require that. Like I described, IMO it does make sense to allow different values, and also to allow using this opcode on both group level and region level, don't you think?

Regarding your latest patch version: Looks better now, however there is definitely one show stopper yet; you are using a st::multimap for code that is executed by the audio thread. All code executed by the audio thread must be real-time safe, which prohibits any kind of system calls. STL containers classes do allocate memory on demand, which implies system calls that may take very long to complete and hence can cause audio dropouts. For that reason we have real-time safe variants of various standard STL classes. In this particular case our RTAVLTree class would be an appropriate replacement for the std::multimap:

http://svn.linuxsampler.org/cgi-bin/viewvc.cgi/linuxsampler/trunk/src/common/RTAVLTree.h?view=markup