Electronics for sensing partial

[bsantero]

So I've had it in my head to create a midi trombone for some time now, and there are the obvious EWI and Yamaha digital trumpet things.

Does anyone have any experience with electronics for sensing the partial you're currently playing, i.e. pitch recognition through mic and waveform interpolation, or air pressure module, combination of both?

The slide doesn't seem to be such a problem, just distance measuring.

Any engineers in the house?

[afugate]

How do you plan to make the horn slot to produce the partials?

--Andy in OKC

[mbarbier]

Check out Jeff Synder's feedback trombone project. If you google it some article's he's written will come up. We've gone through a few sets of pretty serious rebuilds since then (and have more planned) but it gives the basic gist.

It's driven by a breath sensor that initiates the feedback then filter controlled via joystick (roughly where the 2nd paddle for a bass trombone is) which allows you to shift it up and down the overtone series (or whatever else you code it to do though that's less predictable because of the horns acoustics).

The version in the same articles uses a string potentiometer to measure slide. New version uses a laser, which seems to work pretty well and it much less cumbersome than the string pot.

The project is a little extra but i think it will lead to some interesting results long term.

[bsantero]

How do you plan to make the horn slot to produce the partials?

--Andy in OKC

This is my question as well! I don't particularly care about the feel of the slotting as long as the software will be switchable to...partialize? or continuous gliss as if there no harmonics and it were just a mouthpiece.

I assume a breath sensor/microphone would be utilized to filter the signal and translate it to midi (which is my ultimate goal)

Check out Jeff Synder's feedback trombone project. If you google it some article's he's written will come up. We've gone through a few sets of pretty serious rebuilds since then (and have more planned) but it gives the basic gist.
...

This is quite interesting and what a fantastic writeup! I'll read through it, thanks for sharing this.
https://www.researchgate.net/publicatio ... nstruments (for those interested)

[ExZacLee]

Check out all the available information on the development of the EVI (Electronic Valve Instrument) - Nyle Steiner (inventor of EWI and EVI if I remember right), John Swanna and "Uncle" John Diversa (and many others) have been involved with this. Steiner's most recent build looks pretty amazing. I forget what Diversa said about his, I remember there being octave switches and maybe a partial selector? Steiner has been working on an embouchure pressure sensor that looks very promising....

https://www.patchmanmusic.com/NyleMIDIEVI.html

google "Electronic Valve Instrument"

There is a very in depth thesis about the EVI by Ronald Cole that you may be able to find online - I have it but I can't distribute it as I'm not sure about the copyright situation with it. It is out there if you want to find it. Ronald Cole's thesis (1998) is probably the most in depth paper on the topic available to date (unless there are others I haven't seen, which is possible as this is a 20 year old paper.)

[MrHCinDE]

I wrote a Master’s Thesis on polyphonic real-time note recognition using a basic microphone connected to the mic. input on a standard Windows laptop. Monophonic worked pretty reliably with a piano, polyphonic was tricky trying to unravel overlapping overtones. During the development I played around with trombone and it worked ok too.

This was back in 2004/2005 so presumably things have moved on a bit but my concept was:

1) Sound capture using Microsoft DirectSound
2) Read sound capture buffer
3) Apply ‘Constant Q’ transform (neat way of handling overtone series)
4) Recognize ‘best-fit’ of transformed data to pre-recorded profile of instrument by cross-correlation
5) For polyphonic, use ‘Clean’ algorithm to remove recognised note from data and repeat Step 4 until no more notes above minimum threshold
6) Detect start/end of notes and write to midi file

Having a profile for the instrument really helped, i.e. play all notes in range and use for pattern recognition.

Also, Handling the different decay rates of each overtone isn’t easy.

At the time I wrote it in C using the DirectX SDK.

Maybe one day I’ll update the sound capture part for iPhone or something and try to adjust the algorithm to work better with trombone.

[quiethorn]

I had an EWI about 15 years ago and thought about how I'd design a version with a slide. I figured a laser sensor for measuring slide distance would be best for the slide. For the partials, I figured why bother with some kind of trombone-like mouthpiece and fuss about trying to sense partials from some weird fake mouthpiece buzz; instead, use a breath sensor like an EWI and come up with some way to switch partials with buttons or rollers like an EWI. The difference with EWI is that each roller is 1 octave, where as with trombone, each partial only gets you a tritone of range, and the partials get closer as you go higher, so you'd need more than the 8 or 9 rollers on an EWI otherwise you're limited to high Bb. But above high Bb, things start getting close together, so having 1 button/roller per partial gets redundant. So I figured maybe just take partials 4 - 7, make 1 button for each, and then have several octave rollers to adjust those 4 - 7 partials up and down by octaves. It'd take some getting used to, for sure, but then again, so does EWI... and acoustic trombone for that matter!

You could also have different slide profiles to fit the different known intonation peculiarities of certain horns. You could also have a profile where 7th partial is "in-tune"... usable Ab in 1st position!

Also, I'd forgo doing any of this in MIDI, or at least make it able to output MIDI and CV. The whole MIDI note on/off plus pitch bend thing would fall apart pretty quick on ETI (electronic trombone instrument). Each note on would have to also measure the slide distance and apply a MIDI pitch bend, which is probably fine for legit playing but would break down pretty fast for anyone who glisses a lot (I think pitch bend is 13-bits maybe, so you get 8192 values in either direction, so maybe you could just make ALL note-ons trigger the same note in each octave and apply a pitch bend based on the partial selected and slide distance?). But the benefit of MIDI is you can trigger samples of other instruments, so there's that. CV, on the other hand, would get you way more flexibility and "natural" feeling, but you'd lose out on being able to trigger samples.. but you could plug into some of these Eurorack synths and do really wild stuff.

It'd also be cool if you could do this all over OSC so it could be wireless.

[quiethorn]

I wrote a Master’s Thesis on polyphonic real-time note recognition using a basic microphone connected to the mic. input on a standard Windows laptop. Monophonic worked pretty reliably with a piano, polyphonic was tricky trying to unravel overlapping overtones. During the development I played around with trombone and it worked ok too.

This was back in 2004/2005 so presumably things have moved on a bit but my concept was:

1) Sound capture using Microsoft DirectSound
2) Read sound capture buffer
3) Apply ‘Constant Q’ transform (neat way of handling overtone series)
4) Recognize ‘best-fit’ of transformed data to pre-recorded profile of instrument by cross-correlation
5) For polyphonic, use ‘Clean’ algorithm to remove recognised note from data and repeat Step 4 until no more notes above minimum threshold
6) Detect start/end of notes and write to midi file

Having a profile for the instrument really helped, i.e. play all notes in range and use for pattern recognition.

Also, Handling the different decay rates of each overtone isn’t easy.

At the time I wrote it in C using the DirectX SDK.

Maybe one day I’ll update the sound capture part for iPhone or something and try to adjust the algorithm to work better with trombone.

I think I read that Rodney Oakes uses/used one of the Roland pitch-to-MIDI boxes for this.
https://www.rodoakescomposerandtrombone.com/

[MrHCinDE]

Thanks for the link to Rodney Oakes, interesting stuff!

Could there be scope to combine some measuring device for slide position and an audio-based recognition of the partial using a Raspberry Pi? I'm pretty sure if my old laptop from 2004/5 had enough processing power the Raspberry Pi would be up to the job. Here's a possible concept:

Slide position measurement using a range finder sensor, maybe something like this:
https://www.robotshop.com/en/tof-range- ... 53l1x.html
https://pypi.org/project/VL53L1X/0.0.1/

Audio input over USB sound card, depending on how much of the trombone you want to hear, maybe with Silent Brass mute mic.

Recognition of 'most-likely' note based slide position from position sensor and best-fit partial extracted from FFT of audio stream. I'd be interested to try using the slide position for adjustment of the note recognition algorithm so that only valid partials can be recognised for a given length of air column, taking into account a physical model (digital replica?) of the partial series. This could be a way to handle effects like pitch-bend and out-of-tune partials. The phyiscal model could be fine-tuned by benchmarking the instrument in advance. It would be quite straightforward to even have a pedal that switches over between fitting to nearest note or allowing pitch-bend etc. as an additional input to the Raspberry Pi.

Maybe you could also pick up some more characteristics from analyzing the audio stream such as amplitude and decay.

If you want to go with MIDI, setting up the interface seems relatively strightforward:
https://www.instructables.com/id/PiMiDi ... earned-to/

As others have said, I have my doubts whether MIDI is really the best way to communicate what you can do with a trombone.

[AndrewMeronek]

Also, I'd forgo doing any of this in MIDI, or at least make it able to output MIDI and CV. The whole MIDI note on/off plus pitch bend thing would fall apart pretty quick on ETI (electronic trombone instrument). Each note on would have to also measure the slide distance and apply a MIDI pitch bend, which is probably fine for legit playing but would break down pretty fast for anyone who glisses a lot (I think pitch bend is 13-bits maybe, so you get 8192 values in either direction, so maybe you could just make ALL note-ons trigger the same note in each octave and apply a pitch bend based on the partial selected and slide distance?). But the benefit of MIDI is you can trigger samples of other instruments, so there's that. CV, on the other hand, would get you way more flexibility and "natural" feeling, but you'd lose out on being able to trigger samples.. but you could plug into some of these Eurorack synths and do really wild stuff.

Technically, MIDI should be able to handle a tritone pitch bend range. I found this blurb on using MIDI pitch bend script: https://sites.uci.edu/camp2014/2014/04/ ... -messages/

but it says "The amount of alteration in pitch caused by the pitchbend value is determined by the receiving device (i.e., the synthesizer or sampler)."

I'd recommend to emulate a trombone gliss, set the range to +/- a minor third plus a small amount (maybe have a mechanism to reduce that range for trigger registers) and default to the highest possible pitch bend for 1st position. Then, you're just subtracting as the slide goes out and there's no unused pitch bend range.

[quiethorn]

Thanks for the link to Rodney Oakes, interesting stuff!

Could there be scope to combine some measuring device for slide position and an audio-based recognition of the partial using a Raspberry Pi? I'm pretty sure if my old laptop from 2004/5 had enough processing power the Raspberry Pi would be up to the job. Here's a possible concept:

Slide position measurement using a range finder sensor, maybe something like this:
https://www.robotshop.com/en/tof-range- ... 53l1x.html
https://pypi.org/project/VL53L1X/0.0.1/

Audio input over USB sound card, depending on how much of the trombone you want to hear, maybe with Silent Brass mute mic.

Recognition of 'most-likely' note based slide position from position sensor and best-fit partial extracted from FFT of audio stream. I'd be interested to try using the slide position for adjustment of the note recognition algorithm so that only valid partials can be recognised for a given length of air column, taking into account a physical model (digital replica?) of the partial series. This could be a way to handle effects like pitch-bend and out-of-tune partials. The phyiscal model could be fine-tuned by benchmarking the instrument in advance. It would be quite straightforward to even have a pedal that switches over between fitting to nearest note or allowing pitch-bend etc. as an additional input to the Raspberry Pi.

Maybe you could also pick up some more characteristics from analyzing the audio stream such as amplitude and decay.

If you want to go with MIDI, setting up the interface seems relatively strightforward:
https://www.instructables.com/id/PiMiDi ... earned-to/

As others have said, I have my doubts whether MIDI is really the best way to communicate what you can do with a trombone.

You could probably send the slide measurements wirelessly over OSC with a Huzzah32 (https://www.adafruit.com/product/3405), use Pure Data to covert the OSC to whatever you want. It's small enough that you can strap it to your horn. I use one to trigger audio input for my ambient music:


Then run the audio through a Raspberry Pi, output CV for pitch and amplitude, have a couple buttons on the Huzzah32 for octave shifting, connect into a Eurorack oscillator and VCA and do whatever.

Regardless of any of this, I think a project that simply measures slide distance would still be neat. You could have slide distance set different parameters on effects on the acoustic horn. Slide distance could change delay time on a delay VST, change pitch shift, change reverb tail length, ring modulator frequency, etc., so that playing a middle D in 1st, 4th, and 7th position would all sound different.