Discussion:
My second set of EXPERIMENTAL_MIDI_OUT changes are ready, now with scrubbing support!
(too old to reply)
Robert Hänggi
2017-07-03 06:18:36 UTC
Permalink
Hi Pokechu22

Thanks for your work on MIDI.

I'm less interested in scrubbing than in having fundamental features
ready for 2.2.0.

For a multi-track project, it is essential to be able to make
gain-staging (and pan-staging as a second option).

Hence my questions:
- How do keyboard users change Gain/Volume/Velocity?
- How do I interpret the velocity values in the *.aup files?
Neither do They seem related to the normal data range 0-127 nor to dB
values (-60.0 gives a reduction of 12 dB after recording/rendering).

Roger has some knowledge in the field of Velocity to dB
interpretation. See e.g. his paper
http://quod.lib.umich.edu/i/icmc/bbp2372.2006.042?rgn=main;view=fulltext

So I wonder if we should show
- the normal gain dialog for MIDI on the focused track
- a Velocity dialog (with range 0 - 127) instead.
- both at the same time but dB as informal interpretation.
- an additional control for the mapping velocity-dB (logarithmic or
square root with an optional dynamic range)
- and/or include the latter in the MIDI preferences

Note that we have also two shortcuts for adjusting the gain on the
focused track in 1 dB steps.
I think they should also move the velocity slider.

I assume that the velocity is automatically set to either maximum or mean.

Which one?
BTW, the velocity slider should actually be a volume control--they are
not the same.

Regards
Robert
I've finished up my second round of EXPERIMENTAL_MIDI_OUT changes.
- 5e2e2d4cdb "Add more method and field documentation": General
documentation improvements, which I used to first get an understanding
of
the code. No actual logic changes.
Removes unused methods (some of which were already commented out)
- f5645b5689 "Fix drawing MIDI channels with IDs greater than 16": A bug
fix for MIDI files that have more than 16 channels (which can happen via
allegro and certain other MIDI files).
- d313ef7ef5 "Re-add automake files for portmidi", 8a587e7a5e "Attempt
to add the rest of the makefile for portmidi", 9a87587028 "Regenerate
configure scripts and Makefile.in's": An attempt at fixing building for
portmidi. I can't say whether this actually works (I think it doesn't),
but it is at least a starting point that someone who actually
understands
automake (or has a computer fast enough to mess with this without
waiting
an hour to see what was wrong; my pi is definitely suboptimal for
testing).
- 45ddc27589 "Add MIDI device diagnostics": Adds an option under Help ->
Diagnostics to view MIDI device info, similar to the existing option for
audio devices.
- 1308a0486e "Fix velocity offset edge case": Bug fix for a potential
edge case with the velocity slider.
- 0ffd7a88b5 "Fix cut preview with note tracks": Fixes and enables cut
preview for note tracks.
Makes use of the mouse wheel with the vertical ruler for note tracks
(the
piano). Behavior's the same as with normal wave tracks: control zooms
in
and out, and shift scrolls up and down.
- 2113b798e7 "Implement Shift+Right-click for full extent on the note
holding shift to show the full extent.
- 8d470e0821 "Fix note track status message": corrects a typo in the
status message for the note track VRuler (and just makes the message the
same as the one for wave tracks).
- 0251bc0277 "Use mTimeTrack timing rather than mMidiPlaySpeed": Time
track implementation, same as the one that was previously used, except
now
it's stable and behaves correctly
what it says on the tin. This commit is separate since it may be
beneficial to skip or revert it (more on this later).
- d8ef0bcce3 "Get scrubbing to work with MIDI playback": This actually
fixes scrubbing with MIDI. See the commit for details on the change and
potential caveats.
- d144edcef2 "Update documentation for MIDI with time tracks and
scrubbing": A documentation update for some of the previous changes
- 6abac91a11 "Remove MIDI-only playback (without portaudio) logic": This
is probably the most important commit. It drops out the code for
playing
note tracks without using portaudio timing; instead, a portaudio stream
is
always started. This greatly helps with synchronization; for instance,
it
ensures that recording syncs up with MIDI (tested via playing a ~6
minute
long file via a loopback cable; without this commit, it desyncs by the
end,
but with it, it remains in sync the entire way). It also just reduces
duplicate code, meaning that e.g. scrubbing logic only happens in one
place. The actual change here is just one line: changing 'if(
playbackTracks.size() > 0 )' to 'if (playbackTracks.size() > 0 ||
midiPlaybackTracks.size() > 0)' in AudioIO::StartStream (the rest of the
changes are documentation and removing the previous special-case code).
There are still a few bugs that need to be fixed, but I feel like these
changes are all ready to go and everything else should be fairly simple bug
- Scrubbing doesn't work with only a note track; time just doesn't
advance. (This is somewhat better than when scrubbing was completely
ignored; my guess as to why this is happening is that some assumption
about
wave tracks is being violated, for instance maxLen is always 0 in
audacityAudioCallback).
- The play button doesn't become unpressed when the end of the file is
reached when there's only a note track; stop must manually be clicked.
Both of these are in parts of the code which I'm not fully familiar with,
so if someone else has an idea of how to handle it, that'd be helpful (I'll
take a look later today too though).
- Scrubbing with MIDI does not work in reverse. Notes simply are not
played again if they've already been played; this was too complicated
for
me to implement at the time.
I do not attempt to make this clearer within the UI; playback can be
attempted backwards and the notes just won't occur. What to do with
this
will vary; perhaps we could simply block scrubbing backwards with MIDI
for
now?
All of this code is on my midi-out-2
<https://github.com/pokechu22/audacity/tree/midi-out-2> branch.
--Poke
Roger Dannenberg
2017-07-03 14:48:52 UTC
Permalink
Velocity in MIDI is really tricky. A recent model Alesis keyboard I have
offers a choice of velocity curves, and I found the default is almost
unusable with a simple software synthesizer, which I think is using a
widely used "standard": Quicktime MIDI. This means that recorded MIDI
can be virtually unplayable unless you can map velocity to the
synthesizer at hand. I haven't tried measuring recent synthesizers, but
my paper considered some built-in MIDI synthesizers on Windows, so I bet
the model is still reasonable. The main conclusions are that:

- most synthesizers have an approximately quadratic relationship between
MIDI velocity and RMS (in particular, treating velocity like dB with a
log relationship is *not* how synthesizers commonly interpret velocity).

- there's basically one parameter, the dynamic range from velocity 1 to
velocity 127, that should be used to fit the model to any particular
synthesizer (or in some cases to a particular program or patch).

Therefore, if you want to mess with velocity, it would be nice to have a
control for "dynamic range" that ranges from about -50dB to +50dB. What
this would do, on playback, is either expand or compress the velocities
according to the models in the paper. Strictly following the paper,
you'd need to know the dynamic range (r_dB) of the incoming MIDI and the
dynamic range (r_dB) of the outgoing MIDI. I haven't done the math but I
think just the difference does not completely specify the mapping, but
hopefully there's a good solution with only one parameter. I'd be happy
to work on the math and make a more informed recommendation.

As for velocity sliders: yes, it seems a common thing to do is just use
a velocity offset. Synthesizers have a separate "volume pedal" control,
but again the interpretation is up to the synthesizer, which might even
choose to ignore it, and the volume pedal control can be used for
expression within the MIDI file. Most synthesizers, hardware or
software, have their own volume knobs or they go through a mixer, so
trying to implement faders labeled "dB" on the MIDI stream rather than
the synth output will just create surprises, confusion, and misconceptions.

-Roger
I haven't had time to fully think this through (it's late for me), but
* For keyboard controls: I definitely need to look into that; none
are currently implemented (for both the slider and the channel
controls).
* The slider functions as a velocity /offset/, adjusting the values
in the actual track (it doesn't specify values for all the
notes). So, the units are still the same as normal MIDI velocity
units; it just changes the existing value. I /think/ (haven't had
a chance to read the paper you linked) that this would make it
harder to convert to dB, but I don't know for sure. The default
value for the slider is 0: no change from the actual values in the
MIDI file.
--Poke
On Sun, Jul 2, 2017 at 11:18 PM, Robert HÀnggi
Hi Pokechu22
Thanks for your work on MIDI.
I'm less interested in scrubbing than in having fundamental features
ready for 2.2.0.
For a multi-track project, it is essential to be able to make
gain-staging (and pan-staging as a second option).
- How do keyboard users change Gain/Volume/Velocity?
- How do I interpret the velocity values in the *.aup files?
Neither do They seem related to the normal data range 0-127 nor to dB
values (-60.0 gives a reduction of 12 dB after recording/rendering).
Roger has some knowledge in the field of Velocity to dB
interpretation. See e.g. his paper
http://quod.lib.umich.edu/i/icmc/bbp2372.2006.042?rgn=main;view=fulltext
<http://quod.lib.umich.edu/i/icmc/bbp2372.2006.042?rgn=main;view=fulltext>
So I wonder if we should show
- the normal gain dialog for MIDI on the focused track
- a Velocity dialog (with range 0 - 127) instead.
- both at the same time but dB as informal interpretation.
- an additional control for the mapping velocity-dB (logarithmic or
square root with an optional dynamic range)
- and/or include the latter in the MIDI preferences
Note that we have also two shortcuts for adjusting the gain on the
focused track in 1 dB steps.
I think they should also move the velocity slider.
I assume that the velocity is automatically set to either maximum or mean.
Which one?
BTW, the velocity slider should actually be a volume control--they are
not the same.
Regards
Robert
I've finished up my second round of EXPERIMENTAL_MIDI_OUT changes.
- 5e2e2d4cdb "Add more method and field documentation": General
documentation improvements, which I used to first get an understanding
of
the code. No actual logic changes.
- a0712c7b72 "Remove unused and unneeded MIDI methods and
Removes unused methods (some of which were already commented out)
- f5645b5689 "Fix drawing MIDI channels with IDs greater than
16": A bug
fix for MIDI files that have more than 16 channels (which can happen via
allegro and certain other MIDI files).
- d313ef7ef5 "Re-add automake files for portmidi", 8a587e7a5e
"Attempt
to add the rest of the makefile for portmidi", 9a87587028 "Regenerate
configure scripts and Makefile.in's": An attempt at fixing
building for
portmidi. I can't say whether this actually works (I think
it doesn't),
but it is at least a starting point that someone who actually
understands
automake (or has a computer fast enough to mess with this without
waiting
an hour to see what was wrong; my pi is definitely suboptimal for
testing).
- 45ddc27589 "Add MIDI device diagnostics": Adds an option
under Help ->
Diagnostics to view MIDI device info, similar to the existing option for
audio devices.
- 1308a0486e "Fix velocity offset edge case": Bug fix for a
potential
edge case with the velocity slider.
- 0ffd7a88b5 "Fix cut preview with note tracks": Fixes and
enables cut
preview for note tracks.
- 71b0ba0e02 "Implement scrolling for the note track vertical
Makes use of the mouse wheel with the vertical ruler for note tracks
(the
control zooms
in
and out, and shift scrolls up and down.
- 2113b798e7 "Implement Shift+Right-click for full extent on
the note
holding shift to show the full extent.
- 8d470e0821 "Fix note track status message": corrects a typo
in the
status message for the note track VRuler (and just makes the message the
same as the one for wave tracks).
- 0251bc0277 "Use mTimeTrack timing rather than
mMidiPlaySpeed": Time
track implementation, same as the one that was previously used, except
now
it's stable and behaves correctly
- 16c55b4fe0 "Enable scrubbing when either type of audio
what it says on the tin. This commit is separate since it may be
beneficial to skip or revert it (more on this later).
- d8ef0bcce3 "Get scrubbing to work with MIDI playback": This
actually
fixes scrubbing with MIDI. See the commit for details on the change and
potential caveats.
- d144edcef2 "Update documentation for MIDI with time tracks and
scrubbing": A documentation update for some of the previous changes
- 6abac91a11 "Remove MIDI-only playback (without portaudio)
logic": This
is probably the most important commit. It drops out the code for
playing
note tracks without using portaudio timing; instead, a
portaudio stream
is
always started. This greatly helps with synchronization; for
instance,
it
ensures that recording syncs up with MIDI (tested via playing
a ~6
minute
long file via a loopback cable; without this commit, it
desyncs by the
end,
but with it, it remains in sync the entire way). It also
just reduces
duplicate code, meaning that e.g. scrubbing logic only
happens in one
place. The actual change here is just one line: changing 'if(
playbackTracks.size() > 0 )' to 'if (playbackTracks.size() > 0 ||
midiPlaybackTracks.size() > 0)' in AudioIO::StartStream (the
rest of the
changes are documentation and removing the previous
special-case code).
There are still a few bugs that need to be fixed, but I feel
like these
changes are all ready to go and everything else should be fairly
simple bug
- Scrubbing doesn't work with only a note track; time just
doesn't
advance. (This is somewhat better than when scrubbing was completely
ignored; my guess as to why this is happening is that some
assumption
about
wave tracks is being violated, for instance maxLen is always 0 in
audacityAudioCallback).
- The play button doesn't become unpressed when the end of
the file is
reached when there's only a note track; stop must manually be clicked.
Both of these are in parts of the code which I'm not fully
familiar with,
so if someone else has an idea of how to handle it, that'd be
helpful (I'll
take a look later today too though).
- Scrubbing with MIDI does not work in reverse. Notes simply
are not
played again if they've already been played; this was too complicated
for
me to implement at the time.
I do not attempt to make this clearer within the UI; playback
can be
attempted backwards and the notes just won't occur. What to
do with
this
will vary; perhaps we could simply block scrubbing backwards
with MIDI
for
now?
All of this code is on my midi-out-2
<https://github.com/pokechu22/audacity/tree/midi-out-2
<https://github.com/pokechu22/audacity/tree/midi-out-2>> branch.
--Poke
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
audacity-devel mailing list
https://lists.sourceforge.net/lists/listinfo/audacity-devel
<https://lists.sourceforge.net/lists/listinfo/audacity-devel>
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
audacity-devel mailing list
https://lists.sourceforge.net/lists/listinfo/audacity-devel
Robert Hänggi
2017-07-03 17:14:51 UTC
Permalink
Thanks Roger

I've just measured some values for the Fluid software synth (over bassmidi).
Unfortunately, I took the Peak instead of the RMS.
The results for patch 000 are at the very end of this post.

The dynamic range that I got is about 81 dB (unusually high).
It is fairly evident that a linear approximation won't do (which
actually means a fit to the logarithmic scale).

It always depends on how many velocity layers have been defined for a patch.
There are soundfonts of some 100 mb that define a single instrument,
e.g. a Steinway with samples for all keys and about 32 velocity
layers.
Thus, a polynomial approximation had to have order 31 in the worst case. ;)

I think that 3 coefficients are likely to be enough.
For the current sample measurement, two coefficients work well, at
least over a certain range.
Polynomial (order 3) for offsets 0 to -112 gives:
4.71e-5 0.00477 0.33 -4.15 (x3+x2+x+c)

I think that we can afford to omit two terms since the rendering of
the song deviates from measurement to measurement by some 1.5 dB.

In general, I would measure three velocity settings (e.g. 127, 79 and
31) and create the mapping from that.

As Roger points out, the translation is very tricky.

The simplest solution is to stick with the velocity values throughout.

Hence, the gain dialog should show -127 to +127 (instead of -36.0 to
+36.0) for a notetrack with focus.
I'm not sure what the step size for increase/decrease should be
(normally 1 dB). Probably also one (not dB) in order to cover jumpy
velocity mappings.

There's another peculiarity with MIDI:
When we change the velocity for all notes, we will eventually end up
with compressed sound (all velocities either 127 or 1).
In other words, the velocity values should actually be stored as
floats in order to make repeated amplification safer.
It is not a problem as long as effects don't work with notetracks.

Robert

(Data Grand Piano)

V-offset Peak (dB)
127 -1.9
24 -3.6
16 -4.0
8 -4.4
0 -5.0
-8 -6.2
-16 -7.6
-24 -9.5
-32 -10.6
-40 -13.2
-48 -15.2
-56 -16.3
-64 -18.5
-72 -21.2
-80 -23.1
-88 -28.1
-96 -32.8
-104 -39.5
-112 -48.1
-120 -62.0
-127 -87.1
Post by Roger Dannenberg
Velocity in MIDI is really tricky. A recent model Alesis keyboard I have
offers a choice of velocity curves, and I found the default is almost
unusable with a simple software synthesizer, which I think is using a
widely used "standard": Quicktime MIDI. This means that recorded MIDI
can be virtually unplayable unless you can map velocity to the
synthesizer at hand. I haven't tried measuring recent synthesizers, but
my paper considered some built-in MIDI synthesizers on Windows, so I bet
- most synthesizers have an approximately quadratic relationship between
MIDI velocity and RMS (in particular, treating velocity like dB with a
log relationship is *not* how synthesizers commonly interpret velocity).
- there's basically one parameter, the dynamic range from velocity 1 to
velocity 127, that should be used to fit the model to any particular
synthesizer (or in some cases to a particular program or patch).
Therefore, if you want to mess with velocity, it would be nice to have a
control for "dynamic range" that ranges from about -50dB to +50dB. What
this would do, on playback, is either expand or compress the velocities
according to the models in the paper. Strictly following the paper,
you'd need to know the dynamic range (r_dB) of the incoming MIDI and the
dynamic range (r_dB) of the outgoing MIDI. I haven't done the math but I
think just the difference does not completely specify the mapping, but
hopefully there's a good solution with only one parameter. I'd be happy
to work on the math and make a more informed recommendation.
As for velocity sliders: yes, it seems a common thing to do is just use
a velocity offset. Synthesizers have a separate "volume pedal" control,
but again the interpretation is up to the synthesizer, which might even
choose to ignore it, and the volume pedal control can be used for
expression within the MIDI file. Most synthesizers, hardware or
software, have their own volume knobs or they go through a mixer, so
trying to implement faders labeled "dB" on the MIDI stream rather than
the synth output will just create surprises, confusion, and misconceptions.
-Roger
I haven't had time to fully think this through (it's late for me), but
* For keyboard controls: I definitely need to look into that; none
are currently implemented (for both the slider and the channel
controls).
* The slider functions as a velocity /offset/, adjusting the values
in the actual track (it doesn't specify values for all the
notes). So, the units are still the same as normal MIDI velocity
units; it just changes the existing value. I /think/ (haven't had
a chance to read the paper you linked) that this would make it
harder to convert to dB, but I don't know for sure. The default
value for the slider is 0: no change from the actual values in the
MIDI file.
--Poke
On Sun, Jul 2, 2017 at 11:18 PM, Robert Hänggi
Hi Pokechu22
Thanks for your work on MIDI.
I'm less interested in scrubbing than in having fundamental features
ready for 2.2.0.
For a multi-track project, it is essential to be able to make
gain-staging (and pan-staging as a second option).
- How do keyboard users change Gain/Volume/Velocity?
- How do I interpret the velocity values in the *.aup files?
Neither do They seem related to the normal data range 0-127 nor to dB
values (-60.0 gives a reduction of 12 dB after recording/rendering).
Roger has some knowledge in the field of Velocity to dB
interpretation. See e.g. his paper
http://quod.lib.umich.edu/i/icmc/bbp2372.2006.042?rgn=main;view=fulltext
<http://quod.lib.umich.edu/i/icmc/bbp2372.2006.042?rgn=main;view=fulltext>
So I wonder if we should show
- the normal gain dialog for MIDI on the focused track
- a Velocity dialog (with range 0 - 127) instead.
- both at the same time but dB as informal interpretation.
- an additional control for the mapping velocity-dB (logarithmic or
square root with an optional dynamic range)
- and/or include the latter in the MIDI preferences
Note that we have also two shortcuts for adjusting the gain on the
focused track in 1 dB steps.
I think they should also move the velocity slider.
I assume that the velocity is automatically set to either maximum or mean.
Which one?
BTW, the velocity slider should actually be a volume control--they are
not the same.
Regards
Robert
I've finished up my second round of EXPERIMENTAL_MIDI_OUT changes.
- 5e2e2d4cdb "Add more method and field documentation": General
documentation improvements, which I used to first get an
understanding
of
the code. No actual logic changes.
- a0712c7b72 "Remove unused and unneeded MIDI methods and
Removes unused methods (some of which were already commented
out)
- f5645b5689 "Fix drawing MIDI channels with IDs greater than
16": A bug
fix for MIDI files that have more than 16 channels (which can
happen via
allegro and certain other MIDI files).
- d313ef7ef5 "Re-add automake files for portmidi", 8a587e7a5e
"Attempt
to add the rest of the makefile for portmidi", 9a87587028
"Regenerate
configure scripts and Makefile.in's": An attempt at fixing
building for
portmidi. I can't say whether this actually works (I think
it doesn't),
but it is at least a starting point that someone who actually
understands
automake (or has a computer fast enough to mess with this
without
waiting
an hour to see what was wrong; my pi is definitely suboptimal
for
testing).
- 45ddc27589 "Add MIDI device diagnostics": Adds an option
under Help ->
Diagnostics to view MIDI device info, similar to the existing
option for
audio devices.
- 1308a0486e "Fix velocity offset edge case": Bug fix for a
potential
edge case with the velocity slider.
- 0ffd7a88b5 "Fix cut preview with note tracks": Fixes and
enables cut
preview for note tracks.
- 71b0ba0e02 "Implement scrolling for the note track vertical
Makes use of the mouse wheel with the vertical ruler for note
tracks
(the
control zooms
in
and out, and shift scrolls up and down.
- 2113b798e7 "Implement Shift+Right-click for full extent on
the note
track VRuler": Implements another behavior that's used with wave
holding shift to show the full extent.
- 8d470e0821 "Fix note track status message": corrects a typo
in the
status message for the note track VRuler (and just makes the
message the
same as the one for wave tracks).
- 0251bc0277 "Use mTimeTrack timing rather than
mMidiPlaySpeed": Time
track implementation, same as the one that was previously used,
except
now
it's stable and behaves correctly
- 16c55b4fe0 "Enable scrubbing when either type of audio
what it says on the tin. This commit is separate since it may
be
beneficial to skip or revert it (more on this later).
- d8ef0bcce3 "Get scrubbing to work with MIDI playback": This
actually
fixes scrubbing with MIDI. See the commit for details on the
change and
potential caveats.
- d144edcef2 "Update documentation for MIDI with time tracks and
scrubbing": A documentation update for some of the previous
changes
- 6abac91a11 "Remove MIDI-only playback (without portaudio)
logic": This
is probably the most important commit. It drops out the code
for
playing
note tracks without using portaudio timing; instead, a
portaudio stream
is
always started. This greatly helps with synchronization; for
instance,
it
ensures that recording syncs up with MIDI (tested via playing
a ~6
minute
long file via a loopback cable; without this commit, it
desyncs by the
end,
but with it, it remains in sync the entire way). It also
just reduces
duplicate code, meaning that e.g. scrubbing logic only
happens in one
place. The actual change here is just one line: changing 'if(
playbackTracks.size() > 0 )' to 'if (playbackTracks.size() > 0
||
midiPlaybackTracks.size() > 0)' in AudioIO::StartStream (the
rest of the
changes are documentation and removing the previous
special-case code).
There are still a few bugs that need to be fixed, but I feel
like these
changes are all ready to go and everything else should be fairly
simple bug
- Scrubbing doesn't work with only a note track; time just
doesn't
advance. (This is somewhat better than when scrubbing was
completely
ignored; my guess as to why this is happening is that some
assumption
about
wave tracks is being violated, for instance maxLen is always 0
in
audacityAudioCallback).
- The play button doesn't become unpressed when the end of
the file is
reached when there's only a note track; stop must manually be
clicked.
Both of these are in parts of the code which I'm not fully
familiar with,
so if someone else has an idea of how to handle it, that'd be
helpful (I'll
take a look later today too though).
- Scrubbing with MIDI does not work in reverse. Notes simply
are not
played again if they've already been played; this was too
complicated
for
me to implement at the time.
I do not attempt to make this clearer within the UI; playback
can be
attempted backwards and the notes just won't occur. What to
do with
this
will vary; perhaps we could simply block scrubbing backwards
with MIDI
for
now?
All of this code is on my midi-out-2
<https://github.com/pokechu22/audacity/tree/midi-out-2
<https://github.com/pokechu22/audacity/tree/midi-out-2>> branch.
--Poke
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
audacity-devel mailing list
https://lists.sourceforge.net/lists/listinfo/audacity-devel
<https://lists.sourceforge.net/lists/listinfo/audacity-devel>
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
audacity-devel mailing list
https://lists.sourceforge.net/lists/listinfo/audacity-devel
Paul Licameli
2017-07-03 18:20:33 UTC
Permalink
On Sun, Jul 2, 2017 at 5:01 PM, Pokechu22 <
I've finished up my second round of EXPERIMENTAL_MIDI_OUT changes.
- 5e2e2d4cdb "Add more method and field documentation": General
documentation improvements, which I used to first get an understanding of
the code. No actual logic changes.
Removes unused methods (some of which were already commented out)
- f5645b5689 "Fix drawing MIDI channels with IDs greater than 16": A
bug fix for MIDI files that have more than 16 channels (which can happen
via allegro and certain other MIDI files).
- d313ef7ef5 "Re-add automake files for portmidi", 8a587e7a5e "Attempt
to add the rest of the makefile for portmidi", 9a87587028 "Regenerate
configure scripts and Makefile.in's": An attempt at fixing building for
portmidi. I can't say whether this actually works (I think it doesn't),
but it is at least a starting point that someone who actually understands
automake (or has a computer fast enough to mess with this without waiting
an hour to see what was wrong; my pi is definitely suboptimal for testing).
- 45ddc27589 "Add MIDI device diagnostics": Adds an option under Help
-> Diagnostics to view MIDI device info, similar to the existing option for
audio devices.
- 1308a0486e "Fix velocity offset edge case": Bug fix for a potential
edge case with the velocity slider.
- 0ffd7a88b5 "Fix cut preview with note tracks": Fixes and enables cut
preview for note tracks.
Makes use of the mouse wheel with the vertical ruler for note tracks (the
piano). Behavior's the same as with normal wave tracks: control zooms in
and out, and shift scrolls up and down.
- 2113b798e7 "Implement Shift+Right-click for full extent on the note
holding shift to show the full extent.
- 8d470e0821 "Fix note track status message": corrects a typo in the
status message for the note track VRuler (and just makes the message the
same as the one for wave tracks).
- 0251bc0277 "Use mTimeTrack timing rather than mMidiPlaySpeed": Time
track implementation, same as the one that was previously used, except now
it's stable and behaves correctly
- 16c55b4fe0 "Enable scrubbing when either type of audio track
exists": what it says on the tin. This commit is separate since it may be
beneficial to skip or revert it (more on this later).
- d8ef0bcce3 "Get scrubbing to work with MIDI playback": This actually
fixes scrubbing with MIDI. See the commit for details on the change and
potential caveats.
- d144edcef2 "Update documentation for MIDI with time tracks and
scrubbing": A documentation update for some of the previous changes
This is probably the most important commit. It drops out the code for
playing note tracks without using portaudio timing; instead, a portaudio
stream is always started. This greatly helps with synchronization; for
instance, it ensures that recording syncs up with MIDI (tested via playing
a ~6 minute long file via a loopback cable; without this commit, it desyncs
by the end, but with it, it remains in sync the entire way). It also just
reduces duplicate code, meaning that e.g. scrubbing logic only happens in
one place. The actual change here is just one line: changing 'if(
playbackTracks.size() > 0 )' to 'if (playbackTracks.size() > 0 ||
midiPlaybackTracks.size() > 0)' in AudioIO::StartStream (the rest of the
changes are documentation and removing the previous special-case code).
There are still a few bugs that need to be fixed, but I feel like these
changes are all ready to go and everything else should be fairly simple bug
- Scrubbing doesn't work with only a note track; time just doesn't
advance. (This is somewhat better than when scrubbing was completely
ignored; my guess as to why this is happening is that some assumption about
wave tracks is being violated, for instance maxLen is always 0 in
audacityAudioCallback).
- The play button doesn't become unpressed when the end of the file is
reached when there's only a note track; stop must manually be clicked.
Both of these are in parts of the code which I'm not fully familiar with,
so if someone else has an idea of how to handle it, that'd be helpful (I'll
take a look later today too though).
- Scrubbing with MIDI does not work in reverse. Notes simply are not
played again if they've already been played; this was too complicated for
me to implement at the time.
I do not attempt to make this clearer within the UI; playback can be
attempted backwards and the notes just won't occur. What to do with this
will vary; perhaps we could simply block scrubbing backwards with MIDI for
now?
All of this code is on my midi-out-2
<https://github.com/pokechu22/audacity/tree/midi-out-2> branch.
--Poke
Thank you for your efforts! I will take time to review and push this week.

I see you have also spared me the trouble of figuring out for myself how to
make it build on Linux. Have you set up for notifications from Travis, the
automated build tool? https://travis-ci.org/

Please do so, and then add one commit to the branch tip, enabling
EXPERIMENTAL_MIDI_OUT unconditionally on all platforms. Then push to your
origin, and await the notification from Travis. Be sure all builds
successfully.

As for "scrubbing," there is that, and there is "seeking." You do the
first by clicking in the scrub bar, and releasing the mouse button, then
moving. You do the second by holding the button as you move. You can hear
the difference in Wave tracks: the first plays back at changed speed, the
second, plays back at unchanged speed but playing many short excerpts of
the track.

We may decide that only the second makes sense for MIDI playback, and that
playing backwards isn't worth attempting.

PRL
------------------------------------------------------------
------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
audacity-devel mailing list
https://lists.sourceforge.net/lists/listinfo/audacity-devel
Loading...