[Audacity-devel] "Stem plot" option for viewing samples

Discussion:

Steve the Fiddle

2017-04-20 12:28:43 UTC

Moving this discussion from GitHub to the devel@ list.

GitHub ref. https://github.com/audacity/audacity/pull/186
Forum ref. http://forum.audacityteam.org/viewtopic.php?f=28&t=94931

Brian Armstrong issued a pull request for a proposed new feature:
When zoomed in to sample level, display samples as a "stem plot".

Here is a screenshot of what it looks like:

This style of visually representing PCM samples is widely used in other
software (particularly 'scientific' DSP type software), including MatLab,
and is arguably more 'truthful' than joining the dots with straight lines
as we do currently.

I'd like to see this as an option in Audacity, subject to a couple of
caveats:

1) I think it should be a non-default 'option', so there would need to be a
way to switch between the current "join the dots" and "stem plot" styles.

It has also been suggested that in the future we could have another 'style'
using sinc interpolation / oversampling to join the dots with a smooth
curve. I'm very much in favour of this option also (to the extent that, if
it works well enough, it could become the default).

2) Ideally, Brian would like stem plots to be used from a relatively low
zoom level, as the horizontal distance between samples becomes greater than
pixel size, but this can cause unpleasant aliasing:

Brian has proposed two methods to avoid aliasing:

a) Restrict zoom levels to factors that give an exact number of pixels per
sample period.

b) Use sub-pixels rendering or anti-aliasing.

Option (a) has a significant down-side, which is that "zoom to fit" fails
if the number of samples selected is not an integer factor of the track
width in samples.

Example: if the track width is 1000 pixels, and 600 samples are selected,
then to fit the selection, we would need sample spacing of 1.67 pixels per
sample, but that will either cause aliasing, or we would zoom to the 600
selected + 400 not selected pixels.In my opinion, breaking "zoom to fit" in
this way is not an acceptable option.

Unfortunately Brian also hit a problem with option (b), which he believes
"may require a fairly substantial amount of work" to get this working
cross-platform.

On the other hand, at zoom levels where we switch from just a line to
"dots", aliasing is hardly, or not at all noticeable, and the Stem Plot
style works nicely.

In order to move the Stem Plot proposal forward, I would like to propose
that we implement Stem Plot as an option, for zoom ranges where aliasing is
not a problem. Extending the feature for Stem Plot at lower zoom levels
could be added later if / when sub-pixels rendering can be made to work
cross-platform.
â
I've created a branch based on Brian's original stem plot and a preference
setting to select this new style. As the Tracks menu was getting rather
long, I've also moved the Tracks Behaviors preferences into a sub-page.

I also fixed a minor bug where stems could overflow from the lower (right)
channel into the upper (left) channel when zooming vertically on the
negative half of the waveform of a stereo track.

A bit of tweaking may be required to enable building on Windows, but it
works for me on Linux (Ubuntu) and Mac (Sierra).
https://github.com/SteveDaulton/audacity/tree/stem-plot

Steve

James Crook

2017-04-20 14:32:54 UTC

Permalink

Are you looking for approval to add this as an option?
I think it is a good/fine option.
What pixel spacing do you reckon is first free from disturbing aliasing?

I am concerned based on experience with the ruler that even 1 sample per
3.3 pixels IF drawn without anti-aliasing will look terrible. Again
based on experience with the ruler we should be able to go to 1 sample
per 2.1 pixels even with crude anti-aliasing which draws each line as
two of different shade, side by side.

Can you show what we will actually get near the most zoomed out stem
plot view?

--James.

Post by Steve the Fiddle
GitHub ref. https://github.com/audacity/audacity/pull/186
Forum ref. http://forum.audacityteam.org/viewtopic.php?f=28&t=94931
When zoomed in to sample level, display samples as a "stem plot".
This style of visually representing PCM samples is widely used in other
software (particularly 'scientific' DSP type software), including MatLab,
and is arguably more 'truthful' than joining the dots with straight lines
as we do currently.
I'd like to see this as an option in Audacity, subject to a couple of
1) I think it should be a non-default 'option', so there would need to be a
way to switch between the current "join the dots" and "stem plot" styles.
It has also been suggested that in the future we could have another 'style'
using sinc interpolation / oversampling to join the dots with a smooth
curve. I'm very much in favour of this option also (to the extent that, if
it works well enough, it could become the default).
2) Ideally, Brian would like stem plots to be used from a relatively low
zoom level, as the horizontal distance between samples becomes greater than
a) Restrict zoom levels to factors that give an exact number of pixels per
sample period.
b) Use sub-pixels rendering or anti-aliasing.
Option (a) has a significant down-side, which is that "zoom to fit" fails
if the number of samples selected is not an integer factor of the track
width in samples.
Example: if the track width is 1000 pixels, and 600 samples are selected,
then to fit the selection, we would need sample spacing of 1.67 pixels per
sample, but that will either cause aliasing, or we would zoom to the 600
selected + 400 not selected pixels.In my opinion, breaking "zoom to fit" in
this way is not an acceptable option.
Unfortunately Brian also hit a problem with option (b), which he believes
"may require a fairly substantial amount of work" to get this working
cross-platform.
On the other hand, at zoom levels where we switch from just a line to
"dots", aliasing is hardly, or not at all noticeable, and the Stem Plot
style works nicely.
In order to move the Stem Plot proposal forward, I would like to propose
that we implement Stem Plot as an option, for zoom ranges where aliasing is
not a problem. Extending the feature for Stem Plot at lower zoom levels
could be added later if / when sub-pixels rendering can be made to work
cross-platform.
â
I've created a branch based on Brian's original stem plot and a preference
setting to select this new style. As the Tracks menu was getting rather
long, I've also moved the Tracks Behaviors preferences into a sub-page.
I also fixed a minor bug where stems could overflow from the lower (right)
channel into the upper (left) channel when zooming vertically on the
negative half of the waveform of a stereo track.
A bit of tweaking may be required to enable building on Windows, but it
works for me on Linux (Ubuntu) and Mac (Sierra).
https://github.com/SteveDaulton/audacity/tree/stem-plot
Steve
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
audacity-devel mailing list
https://lists.sourceforge.net/lists/listinfo/audacity-devel

Steve the Fiddle

2017-04-20 15:20:37 UTC

Permalink

Post by James Crook
Are you looking for approval to add this as an option?
I think it is a good/fine option.
What pixel spacing do you reckon is first free from disturbing aliasing?

I think it's a good option too, but with the difficulties facing what Brian
was 'ideally' wanting to achieve, I was concerned that it may languish as a
pull request for a while and then get left behind, which I think would be a
shame. As you can see, there has been some discussion between Brian and
myself on GitHub, but as this mailing list is the primary forum for
developers, I thought it best to move the discussion here.

In the absence of some kind of anti-aliasing measures, the pixel spacing
used in my branch is, in my opinion, the minimum that is "not too
disturbing. As can be seen in this screenshot, which is at the threshold /
worst case / most zoomed out, the unevenness is noticeable in places, but I
don't personally find it "disturbing".

By the time that we zoom in one click (of the zoom in button) further, it's
not a problem imo, though looking carefully the uneven spacing is still
visible.

â
Some sort of anti-aliasing would clearly be an improvement, but I've seen
the same 'problem' in MatLab without anyone complaining:

â

âSteve

Post by James Crook
I am concerned based on experience with the ruler that even 1 sample per

3.3

Post by James Crook
pixels IF drawn without anti-aliasing will look terrible. Again based on
experience with the ruler we should be able to go to 1 sample per 2.1

pixels

Post by James Crook
even with crude anti-aliasing which draws each line as two of different
shade, side by side.
Can you show what we will actually get near the most zoomed out stem plot
view?
--James.
GitHub ref. https://github.com/audacity/audacity/pull/186
Forum ref. http://forum.audacityteam.org/viewtopic.php?f=28&t=94931
When zoomed in to sample level, display samples as a "stem plot".
This style of visually representing PCM samples is widely used in other
software (particularly 'scientific' DSP type software), including MatLab,
and is arguably more 'truthful' than joining the dots with straight lines
as we do currently.
I'd like to see this as an option in Audacity, subject to a couple of
1) I think it should be a non-default 'option', so there would need to be a
way to switch between the current "join the dots" and "stem plot" styles.
It has also been suggested that in the future we could have another 'style'
using sinc interpolation / oversampling to join the dots with a smooth
curve. I'm very much in favour of this option also (to the extent that, if
it works well enough, it could become the default).
2) Ideally, Brian would like stem plots to be used from a relatively low
zoom level, as the horizontal distance between samples becomes greater than
a) Restrict zoom levels to factors that give an exact number of pixels per
sample period.
b) Use sub-pixels rendering or anti-aliasing.
Option (a) has a significant down-side, which is that "zoom to fit" fails
if the number of samples selected is not an integer factor of the track
width in samples.
Example: if the track width is 1000 pixels, and 600 samples are selected,
then to fit the selection, we would need sample spacing of 1.67 pixels per
sample, but that will either cause aliasing, or we would zoom to the 600
selected + 400 not selected pixels.In my opinion, breaking "zoom to fit" in
this way is not an acceptable option.
Unfortunately Brian also hit a problem with option (b), which he believes
"may require a fairly substantial amount of work" to get this working
cross-platform.
On the other hand, at zoom levels where we switch from just a line to
"dots", aliasing is hardly, or not at all noticeable, and the Stem Plot
style works nicely.
In order to move the Stem Plot proposal forward, I would like to propose
that we implement Stem Plot as an option, for zoom ranges where aliasing is
not a problem. Extending the feature for Stem Plot at lower zoom levels
could be added later if / when sub-pixels rendering can be made to work
cross-platform.
I've created a branch based on Brian's original stem plot and a preference
setting to select this new style. As the Tracks menu was getting rather
long, I've also moved the Tracks Behaviors preferences into a sub-page.
I also fixed a minor bug where stems could overflow from the lower (right)
channel into the upper (left) channel when zooming vertically on the
negative half of the waveform of a stereo track.
A bit of tweaking may be required to enable building on Windows, but it
works for me on Linux (Ubuntu) and Mac (Sierra).
https://github.com/SteveDaulton/audacity/tree/stem-plot
Steve

------------------------------------------------------------------------------

James Crook

2017-04-20 15:41:04 UTC

Permalink

For a feature that is off by default, I think the aliasing in
firsttrack001.png is OK.

If I were RM I would say check it in, and we can work on improving it
using the 'simple' two lines at a time approach (which works fine on
wxDCs). Simple would be OK at that spacing.

However, it isn't my call, and it should be decided on by an RM who has
an idea of when we intend to release and what else is likely to go in to
2.2.0, or else by waiting for a consensus to emerge.

--James.