Cryogenics study

[hyperbolica]

https://www.nytimes.com/2003/11/18/scie ... -flat.html

[BGuttman]

Neat. I hope they keep looking at various aspects of brass instrument manufacture and use.
nobody else seems to be doing this scientifically.

[Doug Elliott]

"There was more different from trumpet to trumpet and player to player that there was from the treatment"

So nothing was proven one way or the other, but they act like it was. Every study I've seen about instrument materials etc is so badly done it would never pass any kind of scrutiny, but they always act like they proved something important.

[BGuttman]

The problem with studies like this is that there is no quantitative result to analyze. We could easily look at the effect of cryo treatment, annealing, or just aging, on something like bass hardness. But when you are looking at a qualitative aspect like "playablity" it becomes very hard.

[hyperbolica]

This is all very interesting to me. Many/most manufacturers use some sort of annealing/quench, and there are contradictory claims. Annealing generally allows the manufacturing stresses (bell hammering) to work themselves out. But quench generally re-induces some stress. "Heat treat" is a generic term that could mean anything at all.

I'm of the camp that thinks that bell material makes a difference in sound or feedback, but it is a fairly small difference. There are a lot of things you can say mathematically about hardness and vibrations, but metal vibration isn't what we hear, it's the vibration of the air column (which can be slightly influenced by the metal vibration).

So we're talking about a 10% effect on a 10% effect. It could be analyzed using wave forms, but again, differences between individuals would cause more variation than differences between testing parameters. Not to say there is no effect, but the effect is small, and maybe only the kind of thing you can detect as feedback to the player rather than direct effect to the listener.

They need musicians and engineers to do these test and really design them properly. And then be ready for very small results.

[brassmedic]

Quenching brass does not harden it, as it does with ferrous metals.

[brassmedic]

I'm not sure why people are saying this "doesn't prove anything". If a process has an effect on an object, then that effect can be noticed. If it is impossible to tell the difference between an object that has undergone the process and an object that has not undergone the process, then the process has had no effect. Whether the study was well done or not is a separate question, but I don't think one can say it's impossible to prove anything in this manner.

[JohnL]

Nowhere near enough detail in the article, but there's this bit:

They played the same sequence on trumpets that had been frozen and those that had not, and then rated the instruments.

I tend to agree with Doug Elliott on this one. Nothing proven one way or the other.

[BGuttman]

It's always harder to prove a negative. The basis of statistical analysis is that a true effect disproves the null hypothesis (that the treatment made no difference).

What we can say from this study is that the benefit of Cryo is not proven. There might be an effect, but this study couldn't find it.

[brassmedic]

Nowhere near enough detail in the article, but there's this bit:

They played the same sequence on trumpets that had been frozen and those that had not, and then rated the instruments.

I tend to agree with Doug Elliott on this one. Nothing proven one way or the other.

It goes on to say, " They were also asked to identify which trumpet matched the sound that ''people say is brighter, freer-blowing or that had more 'presence,' '"

I do agree with Bruce that it would be exceedingly difficult to prove a negative - that there is "no difference at all", but I still think it would be possible to draw certain conclusions. For example, (I'm not saying they did it this way, but we don't know that they didn't): Let's say I gave 10 people 20 trumpets each (10 frozen and 10 not frozen), and asked which ones are brighter in sound, and they don't know which is which, of course. Now if every player picked an equal number from both groups that they say are brighter (5 frozen and 5 unfrozen), it would be reasonable to conclude that the freezing does nothing to change the players' perception of how bright the trumpet sounds. For this purpose, it doesn't matter that the word "bright" is subjective so long as it consistently means the same thing to the player.

Here's a more obvious example: I take 20 trumpets and put the valves in backwards on 10 of them, then ask people to rate which of the 20 sound like a dying donkey. I'd be willing to bet they would correctly separate the group with the backwards valves from the group with the correctly positioned valves, even if they weren't told in advance. Why? Because there would be an easily perceivable difference. So if one group of things has been altered, and there is a perceivable difference from the alteration, it should be possible to identify. If it can't be identified better than random chance, then it's reasonable to assume there is no difference. EDIT: No difference with regard to that parameter, I mean.

[Doug Elliott]

If you're trying to find out whether a treatment makes a difference or not, the SAME instruments need to be tested before and after... With the SAME player.

Take 12 horns, have the player rate each of them and do a high quality recording of all. Then treat half of them and go through the same rating and recording, with nobody knowing which ones were treated or not. That's a double blind study that has some validity.

Do that same study with 12 different players, treating each as its own independent study.

Then you have eliminated or at least accounted for the 2 problems of variation from trumpet to trumpet and player to player. You would have clear documentation as to whether the treatment changed anything, and what the change was, as determined by a variety of players.

[brassmedic]

But according to the article, it takes a couple days to do the treatment. So they would be doing a lot of playing, waiting 2 days, then playing again. I don't think you'd get any kind of consistency between the two phases of the experiment.

[Doug Elliott]

It doesn't matter if there is consistency two days later, because that is accounted for in testing all 12 instruments with the same players. They're either the same or changed. Any inconsistencies would show up uniformly in the untreated instruments, and could easily be accounted for.

And the fact is, there's some inconsistency even playing the same instrument two consecutive times.

[LIBrassCo]

If you're trying to find out whether a treatment makes a difference or not, the SAME instruments need to be tested before and after... With the SAME player.

Take 12 horns, have the player rate each of them and do a high quality recording of all. Then treat half of them and go through the same rating and recording, with nobody knowing which ones were treated or not. That's a double blind study that has some validity.

Do that same study with 12 different players, treating each as its own independent study.

Then you have eliminated or at least accounted for the 2 problems of variation from trumpet to trumpet and player to player. You would have clear documentation as to whether the treatment changed anything, and what the change was, as determined by a variety of players.

I'd just add to get a real trend, up it to over 100 players/instruments. But this would be entirely more relevant and actually produce something potentially worth looking at.

[Doug Elliott]

I think each player needs to test every instrument - twice.
Expecting players to test 12 in a row is already pushing the limit.

[SwissTbone]

I think each player needs to test every instrument - twice.

Yep, before and after. If that part is not done, the research is useless.

[walldaja]

I've got a Sonare trumpet that was treated and I also have a cheap Chinese made cornet that cost the same as the trumpet's mouthpiece. The cornet rivals the trumpet in both sound and playability. I have no idea why the cornet sounds so good and plays so well. Thing is so cheap it doesn't even have a serial number. I didn't buy the trumpet because of the treatment, I don't believe it adds anything. But Doug is right about their faulty methodology. What I think makes a difference with my trumpet it is one of the German made ones, not the later supposedly US made horns. Wouldn't have any of my trombones frozen if it were done for free.

[sungfw]

For those who are interested, the study on which the NYT article was based was published in the Journal of the Acoustical Society of America under the title The acoustic effect of cryogenically treating trumpets," Oct 2013 (attached below).

Note that the authors of the paper DO NOT claim to have "proved" or "disproved" the purported efficacy of cryogenic treatment. Rather, they conclude that performance variation between trumpet-to-trumped and player-to-player, and from session-to-session for any individual player, "overwhelmed" (their term) any effect detectable within the parameters of their experiment:

4.2. Discussion. When comparing treated and untreated trumpets from the same player, data collection session, and player-microphone orientation, no statistically independent results are seen (except for a single data point at one instant in time in a single case). Conversely, significant deviations are seen when different players are compared, and even when different data sets from the same person are compared. Although differences in the mean values of power in the temporal regime are clearly seen in many cases, these variations are overwhelmed by the scatter associated with the data.
A few conclusions can be drawn from this data. First, it may be possible that the cryogenic treatment does have an effect on the timbre and attack of the trumpets, as the large differences in the mean values of frequency content is seen in the time domain. However, the deviation of frequency content from trumpet-to- trumpet overwhelms any effect seen between the two sets of trumpets. Further, the deviation from player-to-player and even session-to-session for the same player also overshadows any effect seen due to the cryogenic treatment. Thus, it is the individual player, and even the current preparedness of the player, that has a greater effect than the cryogenic treatment on the timbre of the trumpets.

5. Conclusion
In many cases, the cryogenically treated trumpets display elevated upper harmonics when compared to their untreated counterparts. This deviation can be seen in both the steady-state and transient regions of the notes played. This could be correlated with the claims that the treatment results in a trumpet with a brighter tone. However, in the case of Player#3, the opposite is seen, with the untreated trumpets displaying stronger upper harmonics (and, presumably, a “darker” tone). In addition to this contradiction found in the data, virtually none of the data is conclusively statically independent. The scatter of data (i.e. variation from trumpet-to-trumpet) overshadows any difference seen between the treated and un- treated trumpets. Further, variations seen between players and between sessions for the same player are also much greater than the variations found between the treated and untreated trumpets. Although it is possible that the cryogenic treatment does have an effect on the timbre of an instrument, the effect is subtle at best when compared to other determining factors.

[JohnL]

If you're trying to find out whether a treatment makes a difference or not, the SAME instruments need to be tested before and after... With the SAME player.

Agreed.

If I were designing the experiment, I would start with a population of untreated trumpets, preferable all brand new instruments of the same make/model produced consecutively. Randomly divide them into three groups and have your players play them, recording data for each individual instrument/player combination (you'll eventually be aggregating the data for each group, but individual data should be preserved). I'd have 'em play the horns a few times over several days, randomizing the order in which the players play them.

Treat the groups as follows:

Group A: Do NOTHING. They never leave the lab.
Group B: Full treatment.
Group C: Everything but the actual freezing (if you're really picky, you even stick 'em in the freeze box at room temp for a while). We do this because there are processes that accompany the cryo treatment (cleaning, disassembly/reassembly, etc.) that we need to eliminate as variables.

Now we go through the playing process (the same number of playing sessions over the same schedule) again and compare the results from the three groups.

When we evaluate the data, we don't just look at aggregate data for each group. We've got a lot of data about players and instruments that we can analyze a lot of ways. For example - maybe some people can tell the difference between treated and untreated and others can't. Maybe some instruments respond to the treatment more or less than others. Maybe there's so much variability in our "do nothing" group that we figure the whole thing is a washout.

I leave it to you to decide what to measure and how to measure it.

[Doubler]

Unless you get the players to play the untreated instruments and then play the same instruments immediately after cryogenic treatment, a valid comparison cannot be made. This, of course, is impossible, as it takes many hours to complete the cooling-heating cycle. A change in the grain structure of metal in a brass instrument will change the way it plays. Whether this change is an improvement is another matter, since grain structure is at best a minor component in the design of a brass instrument.

Generally, cryogenic treatment is claimed to create wear resistance superior to that of non-treated metal. I suspect that different metals respond differently to this treatment. A goal of reducing slide or valve wear over time might be realistic and worthy of pursuit, if the budget allows and the cost-to-improvement factor makes sense.

We can be certain that a cryogenically treated horn will cost more than one that has not had the treatment, regardless of outcome. This is a plus for the company providing this service. Whether a similar benefit to the consumer exists is open to debate.

[brassmedic]

Here's the study, by the way:

http://www.tuftl.tufts.edu/musicenginee ... er_asa.pdf

Yeah, Conn-Selmer definitely wasted their money on that one.

[JohnL]

Yeah, Conn-Selmer definitely wasted their money on that one.

Not entirely. They learned that the study they paid for produced inconclusive results.

[tbonesullivan]

Not entirely. They learned that the study they paid for produced inconclusive results.

Yes, and by looking at discussions, they can see that people believe it works, which makes it a viable option or treatment that they can make money with. So, in that case, it's money well spent.

[timothy42b]

What we can say from this study is that the benefit of Cryo is not proven. There might be an effect, but this study couldn't find it.

No, I think we can say a little more.

There might be an effect, but it is not a large one. If it were it should be detected.

The reason I make that distinction is because in the usual descriptions of the supposed effect, it is perceived to be very very large.

The other thing we can say is that player to player, note to note, day to day variability is really much larger than we realize. That is a result in itself emdash and while we should all know that, it is easy to forget.

[BGuttman]

There is a big placebo effect here. If you did a study where the people knew they were playing modified instruments they might find them better whether they were or not.

I hear too many people who claim the cryo process makes it work better. I'm sure if you want it to it will. I believe the study -- makes no difference (at least that you can tell).

[sungfw]

There is a big placebo effect here. If you did a study where the people knew they were playing modified instruments they might find them better whether they were or not.

Well, maybe.

In the Jones/Rogers study, at least, the players knew that 5 instruments were treated and 5 were untreated, but they did not know which were which, and the testing was conducted double blind, :

such that neither the player nor the researchers were aware if the trumpet under examination had been treated. Players were asked to rate each trumpet on a scale of 1-10 for the categories of tone and playability. They were also asked to record general impressions of each instrument, then guess if the trumpet had been treated based on those attributes that are commonly associated with cryogenic trumpets. Players' opinions of a single trumpet often varied greatly from session to session. The most pronounced came from the professional player. In his first session he identified a particular trumpet as virtually unplayable, stating that he "couldn't imagine anyone liking this horn." In his next session, one week later, he cited this trumpet as being his 2nd favorite of the 10. When guessing which trumpets had been treated, the players as a group were correct 52.5% of the time, with individual players guessing correctly between 40% and 60% over the course of their sessions. (see Chilling Trumpets: Does It Have An Acoustic Effect?)

So the pacebo effect can be accounted for, though perhaps not completely eliminated, in the design of an experiment.

[brassmedic]

Yeah, Conn-Selmer definitely wasted their money on that one.

Not entirely. They learned that the study they paid for produced inconclusive results.

Not sure they did learn that. According to that article, they decided not to offer cryogenic treatment on their instruments based on the study's conclusion that it doesn't do anything. But if you actually read the study, they did find differences between treated and untreated instruments, but concluded that the difference between players is greater than the difference between treated and untreated. I don't think that's really the point. There can be quite profound differences between one player and another; that shouldn't automatically render any change to an instrument useless.

[BGuttman]

The issue is that the cryo process at best makes a very small difference. Probably not cost effective. A customer would be better off sorting through a pile of instruments to find THE one that is best rather than taking one and treating it.

[sungfw]

Not entirely. They learned that the study they paid for produced inconclusive results.

Not sure they did learn that. According to that article, they decided not to offer cryogenic treatment on their instruments based on the study's conclusion that it doesn't do anything. But if you actually read the study, they did find differences between treated and untreated instruments, but concluded that the difference between players is greater than the difference between treated and untreated.

Gee, a NYT reporter misrepresenting a statment of a source. It's not like THAT never happens, does it?

I think Selmer's decision makes perfect sense from an ROI standpoint. Why sink a boatload of money in a process that has negilgible effect? It's not like the cost acquiring, much less maintaining, cryo treatment equipment would have been negligible, particularly on an industrial scale, so, given the negligible possible quantitaive differences detected (well within the statistical variance) between the treated and untreated trumpets, and given the demonstrated inability of players to consistently identify the treated samples, what percentage of buyers could one realistically expect to opt for the treatment, and how long would it take to recoup the initial investment?

[whitbey]

I had an Edwards for an Edwards 261 tenor bell 21 gauge, heat treated, soldered rim, tempered, yellow brass, cryo-treated bell.

The bell is heavy and was slow to respond. I stripped the lacquer. I feel like the cryo thing makes the horn play like a little thinner bell. If I had not traded for this bell and was buying one, I would have just bought a thinner bell gauge rather then cryo freezing this one.

My tech says dent removal is more risky with a cryo horn as the metal is more likely to split.

[brassmedic]

I think Selmer's decision makes perfect sense from an ROI standpoint. Why sink a boatload of money in a process that has negilgible effect? It's not like the cost acquiring, much less maintaining, cryo treatment equipment would have been negligible, particularly on an industrial scale, so, given the negligible possible quantitaive differences detected (well within the statistical variance) between the treated and untreated trumpets, and given the demonstrated inability of players to consistently identify the treated samples, what percentage of buyers could one realistically expect to opt for the treatment, and how long would it take to recoup the initial investment?

Hmmm... when I read their conclusions, I'm not seeing where they say the effect was negligible. They say the effect was overshadowed by the differences between players, level of preparedness, and which session it was. But doesn't that go without saying? It seems quite obvious to me that the difference between an amateur player and a professional player, or between a player who is in shape and a player who is not in shape, is going to be greater than the difference between a treated and untreated bell. I mean, has anyone seriously contended that treating the bell would make an amateur player sound better than a professional player? OF COURSE the effect is going to be subtle compared to these other huge differences. A valid experiment needs to control the variables, not throw them into the mix and say, "Oh, well, I guess this doesn't do anything."

I don't think they proved the effect is negligible; I think they proved nothing. I'm not saying there is a noticeable effect; I'm just saying they didn't prove that there wasn't.

[elmsandr]

Note on these things, when they "prove" things, they are looking to 'reject' or 'fail to reject' the null hypothesis. In this case, the null hypothesis is that there is no discernible difference as the null hypothesis should be a statement of no change. Brad's assertion is correct, that they will not prove there wasn't, but they never really will unless they kinda prove there was a change. That is just how studies like this work.

Lastly, as an engineer in manufacturing, many of the ideas in this thread for 'better' study designs would probably be far worse from a math standpoint. They didn't do a bad job on this study. They weren't using the same horns... but that is kinda the point, you need to try to control some variables (the date of the playing for example) and try to average out the answers. That is a really good way to tease out limited effects. The main change they could have used here was to try larger sample sizes, but that gets expensive.

I still think the largest impediment to any analysis of this is what is the gauge, and what is good/better? If we can't define and agree on the gauge and which way is better, we will never see any decent results on studies like these.

Cheers,
Andy

[sungfw]

Hmmm... when I read their conclusions, I'm not seeing where they say the effect was negligible. They say the effect was overshadowed by the differences between players, level of preparedness, and which session it was. But doesn't that go without saying? It seems quite obvious to me that the difference between an amateur player and a professional player, or between a player who is in shape and a player who is not in shape, is going to be greater than the difference between a treated and untreated bell. I mean, has anyone seriously contended that treating the bell would make an amateur player sound better than a professional player? OF COURSE the effect is going to be subtle compared to these other huge differences. A valid experiment needs to control the variables, not throw them into the mix and say, "Oh, well, I guess this doesn't do anything."

I don't think they proved the effect is negligible; I think they proved nothing. I'm not saying there is a noticeable effect; I'm just saying they didn't prove that there wasn't.

The point of the experiment WAS NOT to "prove" or "disprove" whether or not cryogenic treatment hand an "effect," and (as noted upthread) Jones and Rogers made no such claim to that effect: it was to determine whether or not the effects claimed by proponents of cryo treatment (stress relief, changes to the crystalline structure of the brass, more responsive, richer tone, etc.) were sufficiently demonstrable to justify Conn-Selmer investing in the "technology," because the proponents claim that those effects ARE readily demonstrable.

In order to verify that the treatment has a definitive effect on the trumpets, we approached our research from three distinct perspectives: materials science, quantitative acoustic measurements, and qualitative player responses. Our sample set was comprised of 10 Bach Stradivarius trumpets randomly picked from the Selmer production line, half of which were cryogenically treated. We then enlisted six players of proficiency ranging from beginner to professional, with each participating in between two and four data collection sessions.

The materials science phase of this research found no changes in the crystalline structure of the brass on the microscopic level. Identifying, verifying and quantifying changes in the material is ultimately irrelevant if the trumpet sounds and plays the same after it has been treated. Thus we began looking for differences in acoustic properties between the two sets of trumpets. As scientists, it is not up to us to decide what sounds "good," "warm," or "tubby," (much less to correlate this vernacular with quantitative data). We can identify differences in timbre by measuring frequency content of a sound and graphically displaying what frequencies are present. From this, we get a quantitative description of the sound. We can also compare these graphs of the average frequency amplitudes of the two sets of data and easily calculate the difference in sound. These graphs are especially powerful, because they can tell us the amplitudes of the harmonics (e.g. overtones or partials). Further, we can calculate the frequency content of the steady tone in the middle of the note or look at how the frequency content develops in the beginning, or attack, of a note.

When we compared the average frequency content of the treated trumpets to the untreated set for each player, no conclusions could be drawn regarding the influence of the cryogenic treatment. The amplitudes of the frequencies were scattered greatly, and the related uncertainty in the data was far greater than any differences seen between the two sets of trumpets in all but one isolated case (a particular player playing a particular note, the E5). In this single case the treated trumpets had more power on average in the higher harmonics, indicating a "brighter" or more trebly sound (often cited as a trait of cryogenically treated trumpets).

We repeated the anomalous data set some months later and the difference was not seen again. Interestingly, this player, a student at the New England Conservatory, came in for his first recording session at the end of a summer that was comprised of more sailing than trumpet playing. His second set of data was collected near the end of his senior year, during which he had been practicing many hours a day and performing in public weekly. The overall difference in timbre between the two sets of data was much greater than the difference we had originally seen between the treated and untreated trumpets. Apparently, simply practicing your trumpet does more to change one's tone than does freezing it, even for advanced players. Finally when comparing data from different players, the difference in timbre between some players is even more pronounced.

The most interesting, if not entertaining, data comes from the qualitative results. The trumpets were played and examined in a random, double blind fashion, such that neither the player nor the researchers were aware if the trumpet under examination had been treated. Players were asked to rate each trumpet on a scale of 1-10 for the categories of tone and playability. They were also asked to record general impressions of each instrument, then guess if the trumpet had been treated based on those attributes that are commonly associated with cryogenic trumpets. Players' opinions of a single trumpet often varied greatly from session to session. The most pronounced came from the professional player. In his first session he identified a particular trumpet as virtually unplayable, stating that he "couldn't imagine anyone liking this horn." In his next session, one week later, he cited this trumpet as being his 2nd favorite of the 10. When guessing which trumpets had been treated, the players as a group were correct 52.5% of the time, with individual players guessing correctly between 40% and 60% over the course of their sessions.

– Chilling Trumpets: Does It Have An Acoustic Effect? Popular version of paper 2pMUa6, Presented Tuesday afternoon, November 11, 2003, 146th ASA Meeting, Austin, TX.

So, no measurable change to the crystalline structure, no demonstrable quantitative effect on the sound, no demonstrable qualitative effect on the sound, as judged by the players: I think that for the purposes of the study, that qualifies the effect as "negligible*."

*negligible - so small or unimportant or of so little consequence as to warrant little or no attention

[brassmedic]

The point of the experiment WAS NOT to "prove" or "disprove" whether or not cryogenic treatment hand an "effect," and (as noted upthread) Jones and Rogers made no such claim to that effect: it was to determine whether or not the effects claimed by proponents of cryo treatment (stress relief, changes to the crystalline structure of the brass, more responsive, richer tone, etc.) were sufficiently demonstrable to justify Conn-Selmer investing in the "technology," because the proponents claim that those effects ARE readily demonstrable.

In order to verify that the treatment has a definitive effect on the trumpets, we approached our research from three distinct perspectives: materials science, quantitative acoustic measurements, and qualitative player responses. Our sample set was comprised of 10 Bach Stradivarius trumpets randomly picked from the Selmer production line, half of which were cryogenically treated. We then enlisted six players of proficiency ranging from beginner to professional, with each participating in between two and four data collection sessions.

The materials science phase of this research found no changes in the crystalline structure of the brass on the microscopic level. Identifying, verifying and quantifying changes in the material is ultimately irrelevant if the trumpet sounds and plays the same after it has been treated. Thus we began looking for differences in acoustic properties between the two sets of trumpets. As scientists, it is not up to us to decide what sounds "good," "warm," or "tubby," (much less to correlate this vernacular with quantitative data). We can identify differences in timbre by measuring frequency content of a sound and graphically displaying what frequencies are present. From this, we get a quantitative description of the sound. We can also compare these graphs of the average frequency amplitudes of the two sets of data and easily calculate the difference in sound. These graphs are especially powerful, because they can tell us the amplitudes of the harmonics (e.g. overtones or partials). Further, we can calculate the frequency content of the steady tone in the middle of the note or look at how the frequency content develops in the beginning, or attack, of a note.

When we compared the average frequency content of the treated trumpets to the untreated set for each player, no conclusions could be drawn regarding the influence of the cryogenic treatment. The amplitudes of the frequencies were scattered greatly, and the related uncertainty in the data was far greater than any differences seen between the two sets of trumpets in all but one isolated case (a particular player playing a particular note, the E5). In this single case the treated trumpets had more power on average in the higher harmonics, indicating a "brighter" or more trebly sound (often cited as a trait of cryogenically treated trumpets).

We repeated the anomalous data set some months later and the difference was not seen again. Interestingly, this player, a student at the New England Conservatory, came in for his first recording session at the end of a summer that was comprised of more sailing than trumpet playing. His second set of data was collected near the end of his senior year, during which he had been practicing many hours a day and performing in public weekly. The overall difference in timbre between the two sets of data was much greater than the difference we had originally seen between the treated and untreated trumpets. Apparently, simply practicing your trumpet does more to change one's tone than does freezing it, even for advanced players. Finally when comparing data from different players, the difference in timbre between some players is even more pronounced.

The most interesting, if not entertaining, data comes from the qualitative results. The trumpets were played and examined in a random, double blind fashion, such that neither the player nor the researchers were aware if the trumpet under examination had been treated. Players were asked to rate each trumpet on a scale of 1-10 for the categories of tone and playability. They were also asked to record general impressions of each instrument, then guess if the trumpet had been treated based on those attributes that are commonly associated with cryogenic trumpets. Players' opinions of a single trumpet often varied greatly from session to session. The most pronounced came from the professional player. In his first session he identified a particular trumpet as virtually unplayable, stating that he "couldn't imagine anyone liking this horn." In his next session, one week later, he cited this trumpet as being his 2nd favorite of the 10. When guessing which trumpets had been treated, the players as a group were correct 52.5% of the time, with individual players guessing correctly between 40% and 60% over the course of their sessions.

– Chilling Trumpets: Does It Have An Acoustic Effect? Popular version of paper 2pMUa6, Presented Tuesday afternoon, November 11, 2003, 146th ASA Meeting, Austin, TX.

So, no measurable change to the crystalline structure, no demonstrable quantitative effect on the sound, no demonstrable qualitative effect on the sound, as judged by the players: I think that for the purposes of the study, that qualifies the effect as "negligible*."

*negligible - so small or unimportant or of so little consequence as to warrant little or no attention

That's not what I read. You are quoting a press release which they are calling the "popular version" of the study (whatever that means). Again, the study is here: http://www.tuftl.tufts.edu/musicenginee ... er_asa.pdf

Here are some notable things written:

First,it may be possible that the cryogenic treatment does have an effect on the timbre and attack of the trum-pets,as the large differences in the mean values of frequency content is seen in the time domain.

In many cases,the cryogenically treated trumpets display elevated upper harmonics when compared to their untreated counterparts.This deviation can be seen in both the steady-state and transient regions of the notes played.This could be correlated with the claims that the treatment results in a trumpet with a brighter tone.

O.K., so they didn't prove the absence of any effect, because they admit there might be an effect, right? So, did they prove that the effect was "negligible"? Well, they go on to conclude this:

However,in the case of Player#3, the opposite is seen, with the untreated trumpets displaying stronger upper harmonics (and,presumably, a "darker"tone). In addition to this contradiction found in the data,virtually none of the data is conclusively statically independent . The scatter of data (i.e.variation from trumpet-to-trumpet) overshadow sany difference seen between the treated and un-treated trumpets. Further, variations seen between players and between sessions for the same player are also much greater than the variations found between the treated and untreated trumpets. Although it is possible that the cryogeni treatment does have an effect on the timbre of an instrument, the effect is subtle at best when compared to other determining factors.

Or, from your press release, put into layman's terms:

Apparently, simply practicing your trumpet does more to change one's tone than does freezing it, even for advanced players.

Isn't that patently obvious? If you stop practicing for a couple months, yes there will be a HUGE difference in your sound. So notice what they are saying: they are saying that differences between amateur and professional players, or differences between one player when he is in shape and when he hasn't practiced all summer, are greater than the difference between treated and untreated trumpets. Sure, I can see that, but it doesn't prove a negligible effect, it only proves that OTHER effects are bigger. I think all of us already knew that.

[brassmedic]

I'm also skeptical of this "no changes in the crystalline structure" claim, because it's mentioned in the "popular version" of the study, but not in the study paper itself, and no mention is made of how they determined this. And if they did determine that, why would they say it's "irrelevant" and go ahead with the playing portion of the experiment. If there is literally no physical change to the metal, the study should have been over right then, unless they are studying voodoo.

[JohnL]

They weren't using the same horns... but that is kinda the point, you need to try to control some variables (the date of the playing for example) and try to average out the answers. That is a really good way to tease out limited effects. The main change they could have used here was to try larger sample sizes, but that gets expensive.

But by not using the same instruments, they introduced a significant uncontrolled variable. They could have addressed this by doing a round of playing tests before doing any cryo treatment. As it is, we have no idea if any differences (negligible or not) between the control group and the treatment group are the result of cryo treatment or if they were there before the treatment was applied.

That said, I suspect the study was entirely adequate for Conn-Selmer's needs. If the play testers had unanimously and overwhelming favored the treated instruments, C-S would be looking to offer cryo treatment. Since that was not the case, I expect they'll be leaving cryo treatment to the aftermarket folks.

[brassmedic]

I don't know that "unanimously favoring" a thing should be the criteria though. They offer lacquered trumpets not because they are unanimously favored, but because some players like them. Others prefer silver plate. So they offer both kinds. But I don't think any of the players in the study consistently favored the cryo trumpets, so I agree the choice not to offer them was probably a good one.

[elmsandr]

They weren't using the same horns... but that is kinda the point, you need to try to control some variables (the date of the playing for example) and try to average out the answers. That is a really good way to tease out limited effects. The main change they could have used here was to try larger sample sizes, but that gets expensive.

But by not using the same instruments, they introduced a significant uncontrolled variable. They could have addressed this by doing a round of playing tests before doing any cryo treatment. As it is, we have no idea if any differences (negligible or not) between the control group and the treatment group are the result of cryo treatment or if they were there before the treatment was applied.

That said, I suspect the study was entirely adequate for Conn-Selmer's needs. If the play testers had unanimously and overwhelming favored the treated instruments, C-S would be looking to offer cryo treatment. Since that was not the case, I expect they'll be leaving cryo treatment to the aftermarket folks.

Need to stop thinking about this like a person with a horn and think about it like a person that makes thousands of horns.

You make a widget and they average a 5 on the scale. Whatever scale, whatever widget. You take a subset and you will be fairly certain that they average a 5. Having done this for years, you can set a probability around that (and it should be really high if you know what you are doing). Randomly taking half that sample, you are looking to see if this process changes the average to a 6. Averages start to bring in the Central Limit Theorem. Particularly useful when there are lots of confounding variables. That is what the study is looking to see. I cannot state this enough, particularly with an unreliable gauge, you do NOT care if an individual horn moved from a 4 to a 5 or a 5 to a 6. You need to see the average of a group move and you need to see if the gauge can (again on average) detect the presence of the process. If the gauge cannot reliably detect the process was completed without detailed before and afters on each part, would you pay somebody to do the process? How would you know if they were actually doing it?

[JohnL]

You make a widget and they average a 5 on the scale. Whatever scale, whatever widget. You take a subset and you will be fairly certain that they average a 5. Having done this for years, you can set a probability around that (and it should be really high if you know what you are doing).

Ah, now that's where we're not on the same page. I'm hung up on the question of whether or not the population is sufficiently uniform that the sampling is valid. So many variables...

[BGuttman]

You make a widget and they average a 5 on the scale. Whatever scale, whatever widget. You take a subset and you will be fairly certain that they average a 5. Having done this for years, you can set a probability around that (and it should be really high if you know what you are doing).

Ah, now that's where we're not on the same page. I'm hung up on the question of whether or not the population is sufficiently uniform that the sampling is valid. So many variables...

The problem is the metrics. It's so qualitative and subjective. It becomes nearly impossible to determine if a small improvement was made; and if one was made, will it be noticeable.

[brassmedic]

Let's say you have 10 trumpets and 5 are cryo treated, as they did in the experiment. You test them to see which ones have elevated decibel levels for certain frequencies, i.e. a brighter tone. But maybe some of the trumpets were innately brighter before being treated. The way this experiment could fail is if the cryo treatment does nothing, but by coincidence, the 5 treated trumpets were already brighter than the others, even before they were treated. In that case you would have the false impression that the treatment made them brighter. But the more you increase the number of trumpets tested, the lower the probability that all the treated ones coincidentally were naturally brighter. If you have a large enough sample size, doesn't that mitigate the problem of individual variation? Could get expensive and impractical pretty quickly, though.

[harrisonreed]

So, an inconclusive, sorta botched "experiment" report from 2003 where the researchers admit that player variability negated any meaningful value in the experiment? What am I missing here? Why internet, why?

[BGuttman]

So, an inconclusive, sorta botched "experiment" report from 2003 where the researchers admit that player variability negated any meaningful value in the experiment? What am I missing here? Why internet, why?

What we can say is that the effect of the cryo treatment was either very small or zero. It's certainly not going to replace practice time or suddenly turn you into Bill Watrous, which is probably what the kids are hoping for.

[harrisonreed]

Of course! Why bring up an article from two decades ago now, was my rhetorical question...

[sungfw]

I'm also skeptical of this "no changes in the crystalline structure" claim, because it's mentioned in the "popular version" of the study, but not in the study paper itself

Are you suggesting or accusing Jones of engaging in academic dishonesty?

I have to say, I find this “objection” baffling.

Do you belong to any academic research organizations, and have you ever presented a popular lecture based on a research paper you presented at their annual meetings? I suspect the answer to both questions is “No,” because if you were or had, you would know fully well that papers read at such conferences are published and distributed several months in advance to allow attendees to offer informed critiques, challenge conclusions, and seek clarification on matters of methodology and interpretation during the Q&A period following the presentation, and that in the interim between publication and presentation, authors are not only free, but are expected, to revise those papers in light of feedback from respondents. That's SOP in the hard sciences, soft sciences, and humanities, so, at worst, the fact that the published paper doesn’t mention the mechanical analysis may (probably) reflect(s) an oversight by the presenter: one that would have been address orally during the formal presentation itself or the Q&A following the presentation, and in the text of the popular lecture delivered subsequent to the formal presentation.

, and no mention is made of how they determined this.

You mean, besides employing at least the de minimis standard tools and analytical procedures that would be expected of, and well known to, anyone doing academic research in material science?

And if they did determine that, why would they say it's "irrelevant" and go ahead with the playing portion of the experiment. If there is literally no physical change to the metal, the study should have been over right then, unless they are studying voodoo.

Because the design of the experiment, based on the question posed by Conn-Selmer, required it.

It's called "The Golden Rule of Arts and Sciences," i.e., them that gots the gold makes the rules.

Conn-Selmer paid for the researchers to assess the claims of material, quantitative, and qualitative improvements that were (and still are being) made by advocates of cryo treatment. Consequently, the researchers designed, and subsequently executed, an experiment to investigate all three areas, as per the funding agreement.

I have to say, much the criticism of the study strikes me as being of the proverbial “eunuchs in the harem,” who know how it's done, have seen it done every day, can tell you in excruciating detail how to do it better, but have never done and are unable to do it themselves, variety. Neither the study nor its authors claim to “prove” or “disprove” the efficacy of cryogenic treatment, either as an objective or an outcome; they don’t claim the design of the experiment to be methodologically ideal; nor do they claim its conclusions to be the last word on the subject, so to attack or dismiss the study based on its failure to “prove” something it never set out to “prove,” or to be something that it doesn’t pretend to be, is, at best misguided, and at worst malicious.

Are there methodological issues with the study? From an abstract perspective, sure; but that would be true of ANY experiment (including experiments designed and run by the “experts” here on the forum). But within the constraints of the finite resources (time, money, materials) available to the experimenters and the limited objectives of the experiment, the experiment fulfills it stated objective, not least by providing hard data (as opposed to merely subjective impressions) against which to compare and contrast results of subsequent experiments, and for that reason must be judged to have meaningfully advanced the statis quaestionis.

[elmsandr]

Of course! Why bring up an article from two decades ago now, was my rhetorical question...

Hey, it's still newer than all my horns, so it is still new to them.

Cheers,
Andy

[harrisonreed]

I'm reminded of the "experiments" posted to the old forum by Snorlax ... Snoreswurthers? Snoresworthy! Remember him??

Memories!

[brassmedic]

Do you belong to any academic research organizations, and have you ever presented a popular lecture based on a research paper you presented at their annual meetings? I suspect the answer to both questions is “No,” because if you were or had, you would know fully well that papers read at such conferences are published and distributed several months in advance to allow attendees to offer informed critiques, challenge conclusions, and seek clarification on matters of methodology and interpretation during the Q&A period following the presentation, and that in the interim between publication and presentation, authors are not only free, but are expected, to revise those papers in light of feedback from respondents. That's SOP in the hard sciences, soft sciences, and humanities, so, at worst, the fact that the published paper doesn’t mention the mechanical analysis may (probably) reflect(s) an oversight by the presenter: one that would have been address orally during the formal presentation itself or the Q&A following the presentation, and in the text of the popular lecture delivered subsequent to the formal presentation.

"Oops, we forgot to include our findings that the process we are studying produces no physical changes to the metal."

Yeah, right.

[brassmedic]

I'm reminded of the "experiments" posted to the old forum by Snorlax ... Snoreswurthers? Snoresworthy! Remember him??

Memories!

Or the Schilke "study" where he made a trumpet bell out of lead that produced a perfect sine wave.

[timothy42b]

I'm reminded of the "experiments" posted to the old forum by Snorlax ... Snoreswurthers? Snoresworthy! Remember him??

Memories!

Or the Schilke "study" where he made a trumpet bell out of lead that produced a perfect sine wave.

Yes! A sine wave that was described as sounding so bright that it was unusable. And that was measured on an analog oscilloscope without octave filters.

(I don't think any of the Schilke "studies" were actually done. I think they were "thought experiments." )