Professor A: We're going? OK. Sh - Close your door on {disfmarker} door on the way out? Grad B: OK. Thanks. Professor A: Thanks. Grad B: Oh. Professor A: Yeah. Probably wanna get this other door, too. OK. So. Um. {vocalsound} {vocalsound} What are we talking about today? PhD E: Uh, well, first there are perhaps these uh Meeting Recorder digits that we tested. Professor A: Oh, yeah. That was kind of uh interesting. PhD E: So. Professor A: The {disfmarker} both the uh {disfmarker} {vocalsound} the SRI System and the oth PhD E: Um. Professor A: And for one thing that {disfmarker} that sure shows the {vocalsound} difference between having a lot of uh training data {vocalsound} or not, PhD E: Of data? Yeah. Professor A: uh, the uh {disfmarker} {vocalsound} The best kind of number we have on the English uh {disfmarker} on near microphone only is {disfmarker} is uh three or four percent. PhD E: Mm - hmm. Professor A: And uh it's significantly better than that, using fairly simple front - ends {vocalsound} on {disfmarker} {vocalsound} on the uh {disfmarker} {vocalsound} uh, with the SRI system. PhD E: Mm - hmm. Professor A: So I th I think that the uh {disfmarker} But that's {disfmarker} that's using uh a {disfmarker} a pretty huge amount of data, mostly not digits, of course, but {disfmarker} but then again {disfmarker} Well, yeah. In fact, mostly not digits for the actual training the H M Ms whereas uh in this case we're just using digits for training the H M PhD E: Yeah. Right. Professor A: Did anybody mention about whether the {disfmarker} the SRI system is a {disfmarker} {vocalsound} is {disfmarker} is doing the digits um the wor as a word model or as uh a sub s sub - phone states? PhD E: I guess it's {disfmarker} it's uh allophone models, Professor A: Yeah. Probably. PhD E: so, well {disfmarker} Professor A: Huh? PhD E: Yeah. I think so, because it's their very d huge, their huge system. Professor A: Yeah. PhD E: And. But. So. There is one difference {disfmarker} Well, the SRI system {disfmarker} the result for the SRI system that are represented here are with adaptation. So there is {disfmarker} It's their complete system and {disfmarker} including on - line uh unsupervised adaptation. Professor A: That's true. PhD E: And if you don't use adaptation, the error rate is around fifty percent worse, I think, if I remember. Professor A: OK. PhD E: Yeah. Professor A: It's tha it's that much, huh? PhD E: Nnn. It's {disfmarker} Yeah. It's quite significant. Professor A: Oh. OK. PhD E: Yeah. Professor A: Still. PhD E: Mm - hmm. Professor A: But {disfmarker} but uh what {disfmarker} what I think I'd be interested to do given that, is that we {disfmarker} we should uh {vocalsound} take {disfmarker} I guess that somebody's gonna do this, right? {disfmarker} is to take some of these tandem things and feed it into the SRI system, right? PhD E: Yeah. Professor A: Yeah. PhD E: We can do something like that. Professor A: Yeah. Because {disfmarker} PhD E: Yeah. But {disfmarker} But I guess the main point is the data because uh {vocalsound} I am not sure. Our back - end is {disfmarker} is fairly simple but until now, well, the attempts to improve it or {disfmarker} have fail Ah, well, I mean uh what Chuck tried to {disfmarker} to {disfmarker} to do Professor A: Yeah, but he's doing it with the same data, right? I mean so to {disfmarker} {vocalsound} So there's {disfmarker} there's {disfmarker} there's two things being affected. PhD E: Yeah. So it's {disfmarker} Yeah. Professor A: I mean. One is that {disfmarker} that, you know, there's something simple that's wrong with the back - end. We've been playing a number of states PhD E: Mm - hmm. Professor A: uh I {disfmarker} I don't know if he got to the point of playing with the uh number of Gaussians yet PhD E: Mm - hmm. Professor A: but {disfmarker} but uh, uh, you know. But, yeah, so far he hadn't gotten any big improvement, PhD E: Mm - hmm. Professor A: but that's all with the same amount of data which is pretty small. PhD E: Yeah. Professor A: And um. PhD E: Mmm. So, yeah, we could retrain some of these tandem on {disfmarker} on huge {disfmarker} Professor A: Well, you could do that, but I'm saying even with it not {disfmarker} with that part not retrained, just {disfmarker} just using {disfmarker} having the H M Ms {disfmarker} much better H M PhD E: Ah, yeah. Just {disfmarker} f for the HMM models. Professor A: Yeah. PhD E: Yeah. Mm - hmm. Mm - hmm. Professor A: Um. {vocalsound} But just train those H M Ms using different features, the features coming from our Aurora stuff. PhD E: Yeah. Professor A: So. PhD E: Yeah. But {vocalsound} what would be interesting to see also is what {disfmarker} what {disfmarker} perhaps it's not related, the amount of data but the um recording conditions. I don't know. Because {vocalsound} it's probably not a problem of noise, because our features are supposed to be robust to noise. Professor A: Well, yeah. PhD E: It's not a problem of channel, because there is um {vocalsound} {vocalsound} normalization with respect to the channel. So {disfmarker} Professor A: I {disfmarker} I {disfmarker} I'm sorry. What {disfmarker} what is the problem that you're trying to explain? PhD E: The {disfmarker} the fact that {disfmarker} the result with the tandem and Aurora system are {vocalsound} uh so much worse. Professor A: That the {disfmarker} Oh. So much worse? Oh. PhD E: Yeah. Professor A: I uh but I'm {disfmarker} I'm almost certain that it {disfmarker} it {disfmarker} {vocalsound} I mean, that it has to do with the um amount of training data. PhD E: It {disfmarker} Professor A: It {disfmarker} it's {disfmarker} it's orders of magnitude off. PhD E: Yeah but {disfmarker} Yeah. Yeah but we train only on digits and it's {disfmarker} it's a digit task, so. Well. Professor A: But {disfmarker} but having a huge {disfmarker} If {disfmarker} {vocalsound} if you look at what commercial places do, they use a huge amount of data. PhD E: It {disfmarker} Mm - hmm. Professor A: This is a modest amount of data. PhD E: Alright. Yeah. Professor A: So. {vocalsound} I mean, ordinarily you would say" well, given that you have enough occurrences of the digits, you can just train with digits rather than with, you know" {disfmarker} PhD E: Mm - hmm. Mm - hmm. Professor A: But the thing is, if you have a huge {disfmarker} in other words, do word models {disfmarker} But if you have a huge amount of data then you're going to have many occurrences of similar uh allophones. PhD E: Right. Mmm. Professor A: And that's just a huge amount of training for it. PhD E: Yeah. Professor A: So it's {vocalsound} um {disfmarker} {vocalsound} I {disfmarker} I think it has to be that, because, as you say, this is, you know, this is near - microphone, PhD E: Mm - hmm. Professor A: it's really pretty clean data. PhD E: Mm - hmm. Professor A: Um. Now, some of it could be the fact that uh {disfmarker} let's see, in the {disfmarker} in these multi - train things did we include noisy data in the training? PhD E: Yeah. Professor A: I mean, that could be hurting us actually, for the clean case. PhD E: Yeah. Well, actually we see that the clean train for the Aurora proposals are {disfmarker} are better than the multi - train, Professor A: It is if {disfmarker} Yeah. PhD E: yeah. Professor A: Yeah. Cuz this is clean data, and so that's not too surprising. PhD E: Mm - hmm. Professor A: But um. Uh. So. PhD E: Well, o I guess what I meant is that well, let's say if we {disfmarker} if we add enough data to train on the um on the Meeting Recorder digits, I guess we could have better results than this. Professor A: Uh - huh. Mm - hmm. PhD E: And. What I meant is that perhaps we can learn something uh from this, what's {disfmarker} what's wrong uh what {disfmarker} what is different between TI - digits and these digits and {disfmarker} Professor A: What kind of numbers are we getting on TI - digits? PhD E: It's point eight percent, so. Professor A: Oh. I see. PhD E: Four - Fourier. Professor A: So in the actual TI - digits database we're getting point eight percent, PhD E: Yeah. Yeah. Professor A: and here we're getting three or four {disfmarker} three, let's see, three for this? PhD E: Mm - hmm. Professor A: Yeah. Sure, but I mean, um point eight percent is something like double uh or triple what people have gotten who've worked very hard at doing that. PhD E: Mm - hmm. Professor A: And {disfmarker} and also, as you point out, there's adaptation in these numbers also. So if you, you know, put the ad adap take the adaptation off, then it {disfmarker} for the English - Near you get something like two percent. PhD E: Mmm. Professor A: And here you had, you know, something like three point four. And I could easily see that difference coming from this huge amount of data that it was trained on. PhD E: Mm - hmm. Professor A: So it's {disfmarker} PhD E: Mm - hmm. Professor A: You know, I don't think there's anything magical here. PhD E: Yeah. Professor A: It's, you know, we used a simple HTK system with a modest amount of data. And this is a {disfmarker} a, you know, modern {vocalsound} uh system uh has {disfmarker} has a lot of nice points to it. PhD E: Yeah. Mm - hmm. Professor A: Um. So. I mean, the HTK is an older HTK, even. So. Yeah it {disfmarker} it's not that surprising. PhD E: Mm - hmm. Professor A: But to me it just {disfmarker} it just meant a practical {vocalsound} point that um if we want to {vocalsound} publish results on digits that {disfmarker} that people pay {vocalsound} attention to we probably should uh {disfmarker} Cuz we've had the problem before that you get {disfmarker} show some {vocalsound} nice improvement on something that's {disfmarker} that's uh, uh {disfmarker} it seems like too large a number, and uh {vocalsound} uh people don't necessarily take it so seriously. PhD E: Mm - hmm. Professor A: Um. Yeah. Yeah. So the three point four percent for this uh is {disfmarker} is uh {disfmarker} So why is it {disfmarker} It's an interesting question though, still. Why is {disfmarker} why is it three point four percent for the d the digits recorded in this environment as opposed to {vocalsound} the uh point eight percent for {disfmarker} for {disfmarker} for the original TI - digits database? Um. PhD E: Yeah. th that's {disfmarker} th that's my point Professor A: Given {disfmarker} given the same {disfmarker} Yeah. So ignore {disfmarker} ignoring the {disfmarker} the {disfmarker} the SRI system for a moment, PhD E: I {disfmarker} I {disfmarker} I don't I {disfmarker} Mm - hmm. Professor A: just looking at {vocalsound} the TI - di the uh tandem system, if we're getting point eight percent, which, yes, it's high. It's, you know, it {disfmarker} it's not awfully high, PhD E: Mm - hmm. Professor A: but it's, you know {disfmarker} it's {disfmarker} it's high. Um. {vocalsound} Why is it {vocalsound} uh four times as high, or more? PhD E: Yeah, I guess. Professor A: Right? I mean, there's {disfmarker} {vocalsound} even though it's close - miked there's still {disfmarker} there really is background noise. PhD E: Mm - hmm. Professor A: Um. And {vocalsound} uh I suspect when the TI - digits were recorded if somebody fumbled or said something wrong or something that they probably made them take it over. PhD E: Mm - hmm. Professor A: It was not {disfmarker} I mean there was no attempt to have it be realistic in any {disfmarker} in any sense at all. PhD E: Well. Yeah. And acoustically, it's q it's {disfmarker} I listened. It's quite different. TI - digit is {disfmarker} it's very, very clean and it's like studio recording Professor A: Mm - hmm. PhD E: whereas these Meeting Recorder digits sometimes you have breath noise and Mmm. Professor A: Right. Yeah. So I think they were {disfmarker} PhD E: It's {nonvocalsound} not controlled at all, I mean. Professor A: Bless you. Grad B: Thanks. Professor A: I {disfmarker} Yeah. I think it's {disfmarker} it's {disfmarker} So. Yes. PhD E: Mm - hmm. But Professor A: It's {disfmarker} I think it's {disfmarker} it's the indication it's harder. PhD E: Yeah. Professor A: Uh. {vocalsound} Yeah and again, you know, i that's true either way. I mean so take a look at the uh {disfmarker} {vocalsound} um, the SRI results. I mean, they're much much better, but still you're getting something like one point three percent for uh things that are same data as in T {disfmarker} TI - digits the same {disfmarker} same text. PhD E: Mm - hmm. Professor A: Uh. And uh, I'm sure the same {disfmarker} same system would {disfmarker} would get, you know, point {disfmarker} point three or point four or something {vocalsound} on the actual TI - digits. So this {disfmarker} I think, on both systems the {vocalsound} these digits are showing up as harder. PhD E: Mmm. Professor A: Um. PhD E: Mm - hmm. Professor A: Which I find sort of interesting cause I think this is closer to {disfmarker} uh I mean it's still read. But I still think it's much closer to {disfmarker} to what {disfmarker} what people actually face, {vocalsound} um when they're {disfmarker} they're dealing with people saying digits over the telephone. I mean. {vocalsound} I don't think uh {disfmarker} I mean, I'm sure they wouldn't release the numbers, but I don't think that uh {vocalsound} the uh {disfmarker} the {disfmarker} the companies that {disfmarker} that do telephone {vocalsound} speech get anything like point four percent on their {vocalsound} digits. I'm {disfmarker} I'm {disfmarker} I'm sure they get {disfmarker} Uh, I mean, for one thing people do phone up who don't have uh uh Middle America accents and it's a we we it's {disfmarker} it's {disfmarker} it's US. PhD E: Mm - hmm. Professor A: it has {disfmarker} has many people {vocalsound} {vocalsound} who sound in many different ways. So. Um. I mean. OK. That was that topic. What else we got? PhD E: Um. Professor A: Did we end up giving up on {disfmarker} on, any Eurospeech submissions, PhD E: But {disfmarker} Professor A: or {disfmarker}? I know Thilo and Dan Ellis are {disfmarker} are submitting something, but uh. PhD E: Yeah. I {disfmarker} {vocalsound} I guess e the only thing with these {disfmarker} the Meeting Recorder and, well, {disfmarker} So, I think, yeah {disfmarker} I think we basically gave up. Professor A: Um. {vocalsound} Now, actually for the {disfmarker} for the Aur - uh PhD E: But {disfmarker} Professor A: we do have stuff for Aurora, right? Because {disfmarker} because we have ano an extra month or something. PhD E: Yeah. Yeah. Yeah. So. Yeah, for sure we will do something for the special session. Professor A: Yeah. Well, that's fine. So th so {disfmarker} so we have a couple {disfmarker} a couple little things on Meeting Recorder PhD E: Yeah. Mm - hmm. Professor A: and we have {disfmarker} {vocalsound} We don't {disfmarker} we don't have to flood it with papers. We're not trying to prove anything to anybody. so. That's fine. Um. Anything else? PhD E: Yeah. Well. So. Perhaps the point is that we've been working on {vocalsound} is, yeah, we have put the um the good VAD in the system and {vocalsound} it really makes a huge difference. Um. So, yeah. I think, yeah, this is perhaps one of the reason why our system was not {disfmarker} {vocalsound} not the best, because with the new VAD, it's very {disfmarker} the results are similar to the France Telecom results and perhaps even better sometimes. Professor A: Hmm. Grad B: Huh. PhD E: Um. So there is this point. Uh. The problem is that it's very big and {vocalsound} {vocalsound} we still have to think how to {disfmarker} where to put it and {disfmarker} {vocalsound} um, Professor A: Mm - hmm. PhD E: because it {disfmarker} it {disfmarker} well, this VAD uh either some delay and we {disfmarker} if we put it on the server side, it doesn't work, because on the server side features you already have LDA applied {vocalsound} from the f from the terminal side and {vocalsound} so you accumulate the delay so the VAD should be before the LDA which means perhaps on the terminal side and then smaller {vocalsound} and Professor A: So wha where did this good VAD come from? PhD E: So. It's um from OGI. So it's the network trained {disfmarker} it's the network with the huge amounts on hidden {disfmarker} of hidden units, and um nine input frames compared to the VAD that was in the proposal which has a very small amount of hidden units and fewer inputs. Professor A: This is the one they had originally? PhD E: Yeah. Professor A: Oh. Yeah, but they had to {pause} get rid of it because of the space, didn't they? PhD E: Yeah. So. Yeah. But the abso assumption is that we will be able to make a VAD that's small and that works fine. And. So we can {disfmarker} Professor A: Well. So that's a problem. Yeah. PhD E: Yeah but {disfmarker} nnn. Professor A: But the other thing is uh to use a different VAD entirely. I mean, uh i if {disfmarker} if there's a {vocalsound} if {disfmarker} if {disfmarker} I {disfmarker} I don't know what the thinking was amongst the {disfmarker} the {disfmarker} the {vocalsound} the ETSI folk but um if everybody agreed sure let's use this VAD and take that out of there {disfmarker} PhD E: Mm - hmm. Mm - hmm. They just want, apparently {disfmarker} they don't want to fix the VAD because they think there is some interaction between feature extraction and {disfmarker} and VAD or frame dropping But they still {vocalsound} want to {disfmarker} just to give some um {vocalsound} requirement for this VAD because it's {disfmarker} it will not be part of {disfmarker} they don't want it to be part of the standard. Professor A: OK. PhD E: So. So it must be at least uh somewhat fixed but not completely. So there just will be some requirements that are still not {disfmarker} uh not yet uh ready I think. Professor A: Determined. I see. But I was thinking that {disfmarker} that uh {vocalsound} s" Sure, there may be some interaction, PhD E: Nnn. Professor A: but I don't think we need to be stuck on using our or OGI's {pause} VAD. We could use somebody else's if it's smaller or {disfmarker} PhD E: Yeah. Professor A: You know, as long as it did the job. PhD E: Mm - hmm. Professor A: So that's good. PhD E: Uh. So there is this thing. There is um {disfmarker} Yeah. Uh I designed a new {disfmarker} a new filter because when I designed other filters with shorter delay from the LDA filters, {vocalsound} there was one filter with fif sixty millisecond delay and the other with ten milliseconds Professor A: Right. PhD E: and {vocalsound} uh Hynek suggested that both could have sixty - five sixty - s I think it's sixty - five. Professor A: Yeah. PhD E: Yeah. Both should have sixty - five because {disfmarker} Professor A: You didn't gain anything, right? PhD E: Yeah. And. So I did that and uh it's running. So, {vocalsound} let's see what will happen. Uh but the filter is of course closer to the reference filter. Professor A: Mm - hmm. PhD E: Mmm. Um. Yeah. I think {disfmarker} Professor A: So that means logically, in principle, it should be better. So probably it'll be worse. PhD E: Yeah Professor A: Or in the basic perverse nature uh of reality. Yeah. OK. PhD E: Yeah. Sure. Grad C: Yeah. Professor A: OK. PhD E: Yeah, and then we've started to work with this of um voiced - unvoiced stuff. Professor A: Mm - hmm. PhD E: And next week I think we will {vocalsound} perhaps try to have um a new system with uh uh MSG stream also see what {disfmarker} what happens. So, something that's similar to the proposal too, but with MSG stream. Professor A: Mm - hmm. Mm - hmm. PhD E: Mmm. Professor A: OK. PhD D: No, I w {vocalsound} I begin to play {vocalsound} with Matlab and to found some parameter robust for voiced - unvoiced decision. But only to play. And we {disfmarker} {vocalsound} they {disfmarker} we found that maybe w is a classical parameter, the {vocalsound} sq the variance {vocalsound} between the um FFT of the signal and the small spectrum of time {vocalsound} we {disfmarker} after the um mel filter bank. Professor A: Uh - huh. PhD D: And, well, is more or less robust. Is good for clean speech. Is quite good {vocalsound} for noisy speech. Professor A: Huh? Mm - hmm. PhD D: but um we must to have bigger statistic with TIMIT, Professor A: Mm - hmm. PhD D: and is not ready yet to use on, Professor A: Yeah. PhD D: well, I don't know. Professor A: Yeah. PhD E: Yeah. So, basically we wa want to look at something like the ex the ex excitation signal and {disfmarker} Professor A: Right. PhD D: Mm - hmm. PhD E: which are the variance of it and {disfmarker} PhD D: I have here. I have here for one signal, for one frame. PhD E: Mmm. Professor A: Yeah. Uh - huh. PhD D: The {disfmarker} the mix of the two, noise and unnoise, and the signal is this. Clean, and this noise. Professor A: Uh. PhD D: These are the two {disfmarker} the mixed, the big signal is for clean. Professor A: Well, I'm s uh {disfmarker} There's {disfmarker} None of these axes are labeled, so I don't know what this {disfmarker} What's this axis? PhD D: Uh this is uh {disfmarker} this axis is {vocalsound} nnn," frame" . Professor A: Frame. PhD D: Mm - hmm. Professor A: And what's th what this? PhD D: Uh, this is uh energy, log - energy of the spectrum. Of the this is the variance, the difference {nonvocalsound} between the spectrum of the signal and FFT of each frame of the signal and this mouth spectrum of time after the f may fit for the two, Professor A: For this one. For the noi PhD D: this big, to here, they are to signal. This is for clean and this is for noise. Professor A: Oh. There's two things on the same graph. PhD D: Yeah. I don't know. I {disfmarker} I think that I have d another graph, but I'm not sure. Professor A: So w which is clean and which is noise? PhD E: Yeah. I think the lower one is noise. PhD D: The lower is noise and the height is clean. Professor A: OK. So it's harder to distinguish PhD D: It's height. Professor A: but it {disfmarker} but it g PhD E: Yeah. Professor A: with noise of course but {disfmarker} but {disfmarker} PhD D: Oh. I must to have. Professor A: Uh. PhD D: Pity, but I don't have two different Professor A: And presumably when there's a {disfmarker} a {disfmarker} PhD E: So this should the {disfmarker} the {disfmarker} the t voiced portions. Professor A: Uh - huh. PhD D: Yeah, it is the height is voiced portion. PhD E: The p the peaks should be voiced portion. PhD D: And this is the noise portion. Professor A: Uh - huh. PhD D: And this is more or less like this. But I meant to have see @ @ two {disfmarker} two the picture. Professor A: Yeah. Yeah. PhD D: This is, for example, for one frame. Professor A: Yeah PhD D: the {disfmarker} the spectrum of the signal. And this is the small version of the spectrum after ML mel filter bank. Professor A: Yeah. And this is the difference? PhD D: And this is I don't know. This is not the different. This is trying to obtain {vocalsound} with LPC model the spectrum but using Matlab without going factor and s Professor A: No pre - emphasis? Yeah. PhD D: Not pre - emphasis. Nothing. Professor A: Yeah so it's {disfmarker} doesn't do too well there. PhD D: And the {disfmarker} I think that this is good. This is quite similar. this is {disfmarker} {vocalsound} this is another frame. ho how I obtained the {vocalsound} envelope, {nonvocalsound} this envelope, with the mel filter bank. Professor A: Right. So now I wonder {disfmarker} I mean, do you want to {disfmarker} I know you want to get at something orthogonal from what you get with the smooth spectrum Um. But if you were to really try and get a voiced - unvoiced, do you {disfmarker} do you want to totally ignore that? I mean, do you {disfmarker} do you {disfmarker} I mean, clearly a {disfmarker} a very big {disfmarker} very big cues {vocalsound} for voiced - unvoiced come from uh spectral slope and so on, right? PhD E: Mm - hmm. Professor A: Um. PhD E: Yeah. Well, this would be {disfmarker} this would be perhaps an additional parameter, Professor A: Yeah. PhD E: simply isn't {disfmarker} Professor A: I see. PhD E: Yeah. PhD D: Yeah because when did noise clear {nonvocalsound} in these section is clear PhD E: Uh. Professor A: Mm - hmm. PhD D: if s @ @ {nonvocalsound} val value is indicative that is a voice frame and it's low values Professor A: Yeah. Yeah. Well, you probably want {disfmarker} I mean, {vocalsound} certainly if {vocalsound} you want to do good voiced - unvoiced detection, you need a few features. Each {disfmarker} each feature is {vocalsound} by itself not enough. But, you know, people look at {disfmarker} at slope and {vocalsound} uh first auto - correlation coefficient, divided by power. PhD E: Mmm. Professor A: Or {disfmarker} or uh um there's uh {disfmarker} I guess we prob probably don't have enough computation to do a simple pitch detector or something? I mean with a pitch detector you could have a {disfmarker} {vocalsound} have a {disfmarker} an estimate of {disfmarker} of what the {disfmarker} PhD E: Mmm. Professor A: Uh. Or maybe you could you just do it going through the P FFT's figuring out some um probable {vocalsound} um harmonic structure. Right. And {disfmarker} and uh. PhD E: Mmm. PhD D: you have read up and {disfmarker} you have a paper, {vocalsound} the paper that you s give me yesterday. they say that yesterday {vocalsound} they are some {nonvocalsound} problem PhD E: Oh, yeah. But {disfmarker} Yeah, but it's not {disfmarker} it's, yeah, it's {disfmarker} it's another problem. PhD D: and the {disfmarker} Is another problem. PhD E: Yeah Um. Yeah, there is th this fact actually. If you look at this um spectrum, Professor A: Yeah. PhD E: What's this again? Is it {vocalsound} the mel - filters? PhD D: Yeah like this. Of kind like this. PhD E: Yeah. OK. So the envelope here is the output of the mel - filters Professor A: Mm - hmm. PhD E: and what we clearly see is that in some cases, and it clearly appears here, and the {disfmarker} the harmonics are resolved by the f Well, there are still appear after mel - filtering, Professor A: Mm - hmm. PhD E: and it happens {vocalsound} for high pitched voice because the width of the lower frequency mel - filters {vocalsound} is sometimes even smaller than the pitch. Professor A: Yeah. PhD E: It's around one hundred, one hundred and fifty hertz {vocalsound} Nnn. Professor A: Right. PhD E: And so what happens is that this uh, add additional variability to this envelope and {vocalsound} {vocalsound} um Professor A: Yeah. PhD E: so we were thinking to modify the mel - spectrum to have something that {disfmarker} that's smoother on low frequencies. Professor A: That's as {disfmarker} as a separate thing. PhD E: i Professor A: Yeah. PhD E: Yeah. This is a separate thing. Professor A: Separate thing? PhD D: Yeah. Professor A: Yeah. PhD E: And. Professor A: Yeah. Maybe so. Um. Yeah. So, what {disfmarker} Yeah. What I was talking about was just, starting with the FFT you could {disfmarker} you could uh do a very rough thing to estimate {disfmarker} estimate uh pitch. PhD E: Yeah. Mm - hmm. Professor A: And uh uh, given {disfmarker} you know, given that, uh {vocalsound} you could uh uh come up with some kind of estimate of how much of the low frequency energy was {disfmarker} was explained by {disfmarker} {vocalsound} by uh uh those harmonics. PhD E: Mm - hmm. Professor A: Uh. It's uh a variant on what you're s what you're doing. The {disfmarker} I mean, the {disfmarker} the {vocalsound} the mel does give a smooth thing. But as you say it's not that smooth here. And {disfmarker} and so if you {disfmarker} {vocalsound} if you just you know subtracted off uh your guess of the harmonics then something like this would end up with {vocalsound} quite a bit lower energy in the first fifteen hundred hertz or so and {disfmarker} and our first kilohertz, even. PhD E: Mm - hmm. Professor A: And um {vocalsound} if was uh noisy, the proportion that it would go down would be if it was {disfmarker} if it was unvoiced or something. PhD E: Mm - hmm. Professor A: So you oughta be able to {vocalsound} pick out voiced segments. At least it should be another {disfmarker} another cue. So. {vocalsound} Anyway. PhD E: Mm - hmm. Professor A: OK? That's what's going on. Uh. What's up with you? Grad B: Um {vocalsound} our t I went to {vocalsound} talk with uh Mike Jordan this {disfmarker} this week Professor A: Mm - hmm. Grad B: um {nonvocalsound} and uh {vocalsound} shared with him the ideas about um {vocalsound} extending the Larry Saul work and um I asked him some questions about factorial H M so like later down the line when {vocalsound} we've come up with these {disfmarker} these feature detectors, how do we {disfmarker} {vocalsound} how do we uh {vocalsound} you know, uh model the time series that {disfmarker} that happens um {vocalsound} {vocalsound} and {vocalsound} and we talked a little bit about {vocalsound} factorial H M Ms and how {vocalsound} um when you're doing inference {disfmarker} or w when you're doing recognition, there's like simple Viterbi stuff that you can do for {disfmarker} {vocalsound} for these H M and {vocalsound} the uh {disfmarker} {vocalsound} the great advantages that um a lot of times the factorial H M Ms don't {vocalsound} um {vocalsound} don't over - alert the problem there they have a limited number of parameters and they focus directly on {disfmarker} {vocalsound} on uh the sub - problems at hand so {vocalsound} you can imagine {vocalsound} um {vocalsound} five or so parallel {vocalsound} um features um transitioning independently and then {vocalsound} at the end you {disfmarker} you uh couple these factorial H M Ms with uh {disfmarker} {vocalsound} with uh undirected links um based on {disfmarker} {vocalsound} based on some more data. Professor A: Hmm. Grad B: So he {disfmarker} he seemed {disfmarker} he seemed like really interested in {disfmarker} {vocalsound} in um {disfmarker} in this and said {disfmarker} said this is {disfmarker} this is something very do - able and can learn a lot and um yeah, I've just been {vocalsound} continue reading um about certain things. Professor A: Mm - hmm. Grad B: um thinking of maybe using um {vocalsound} um m modulation spectrum stuff to {vocalsound} um {disfmarker} as features um also in the {disfmarker} in the sub - bands Professor A: Mm - hmm. Grad B: because {vocalsound} it seems like {vocalsound} the modulation um spectrum tells you a lot about the intelligibility of {disfmarker} of certain um words and stuff So, um. Yeah. Just that's about it. Professor A: OK. Grad C: OK. And um so I've been looking at Avendano's work and um uh I'll try to write up in my next stat status report a nice description of {vocalsound} what he's doing, but it's {disfmarker} it's an approach to deal with {vocalsound} reverberation or that {disfmarker} the aspect of his work that I'm interested in the idea is that um {vocalsound} {vocalsound} {vocalsound} normally an analysis frames are um {vocalsound} too short to encompass reverberation effects um in full. You miss most of the reverberation tail in a ten millisecond window and so {vocalsound} {vocalsound} you {disfmarker} you'd like it to be that {vocalsound} um {vocalsound} the reverberation responses um simply convolved um in, but it's not really with these ten millisecond frames cuz you j But if you take, say, a two millisecond {vocalsound} um window {disfmarker} I'm sorry a two second window then in a room like this, most of the reverberation response {vocalsound} is included in the window and the {disfmarker} then it um {vocalsound} then things are l more linear. It is {disfmarker} it is more like the reverberation response is simply c convolved and um {disfmarker} {vocalsound} and you can use channel normalization techniques {vocalsound} like uh in his thesis he's assuming that the reverberation response is fixed. He just does um {vocalsound} mean subtraction, which is like removing the DC component of the modulation spectrum and {vocalsound} that's supposed to d um deal {disfmarker} uh deal pretty well with the um reverberation and um {vocalsound} the neat thing is you can't take these two second frames and feed them to a speech recognizer um {vocalsound} so he does this {vocalsound} um {vocalsound} method training trading the um {vocalsound} the spectral resolution for time resolution {vocalsound} and um {vocalsound} come ca uh synthesizes a new representation which is with say ten second frames but a lower s um {vocalsound} frequency resolution. So I don't really know the theory. I guess it's {disfmarker} these are called" time frequency representations" and h he's making the {disfmarker} the time sh um finer grained and the frequency resolution um less fine grained. PhD E: Mm - hmm. Grad C: s so I'm {disfmarker} I guess my first stab actually in continuing {vocalsound} his work is to um {vocalsound} re - implement this {disfmarker} this thing which um {vocalsound} changes the time and frequency resolutions cuz he doesn't have code for me. So that that'll take some reading about the theory. I don't really know the theory. PhD E: Mm - hmm. Grad C: Oh, and um, {vocalsound} another f first step is um, so the {disfmarker} the way I want to extend his work is make it able to deal with a time varying reverberation response um {vocalsound} and um we don't really know {vocalsound} how fast the um {disfmarker} the reverberation response is varying the Meeting Recorder data um so um {vocalsound} we {disfmarker} we have this um block least squares um imp echo canceller implementation and um {vocalsound} I want to try {vocalsound} finding {vocalsound} the {disfmarker} the response, say, between a near mike and the table mike for someone using the echo canceller and looking at the echo canceller taps and then {vocalsound} see how fast that varies {vocalsound} from block to block. PhD E: Mm - hmm. Grad C: That should give an idea of how fast the reverberation response is changing. PhD E: Mm - hmm. Professor A: OK. Um. I think we're {vocalsound} sort of done. PhD E: Yeah. Professor A: So let's read our digits and go home. Grad C: Um. S so um y you do {disfmarker} I think you read some of the {disfmarker} the zeros as O's and some as zeros. Professor A: Yeah. Grad C: Is there a particular way we're supposed to read them? PhD E: There are only zeros here. Well. Professor A: No." O" {disfmarker}" O" {disfmarker}" O" " O" {disfmarker}" O" {disfmarker}" O" and" zero" are two ways that we say that digit. PhD E: Eee. Yeah. Professor A: So it's {disfmarker} Grad B: Ha! PhD E: But {disfmarker} Professor A: so it's {disfmarker} i PhD E: Perhaps in the sheets there should be another sign for the {disfmarker} if we want to {disfmarker} the {disfmarker} the guy to say" O" or Professor A: No. I mean. I think people will do what they say. PhD E: It's {disfmarker} Professor A: It's OK. PhD E: Yeah. Professor A: I mean in digit recognition we've done before, you have {disfmarker} you have two pronunciations for that value," O" and" zero" . Grad C: Alright. PhD E: OK. Grad C: OK. PhD E: But it's perhaps more difficult for the people to prepare the database then, if {disfmarker} because here you only have zeros Professor A: No, they just write {disfmarker} PhD E: and {disfmarker} and people pronounce" O" or zero {disfmarker} Professor A: they {disfmarker} they write down OH. or they write down ZERO a and they {disfmarker} and they each have their own pronunciation. PhD E: Yeah but if the sh the sheet was prepared with a different sign for the" O" . Professor A: But people wouldn't know what that wa I mean {vocalsound} there is no convention for it. PhD E: OK. Yeah. Professor A: See. I mean, you'd have to tell them {vocalsound}" OK when we write this, say it tha" , PhD E: OK. Professor A: you know, and you just {disfmarker} They just want people to read the digits as you ordinarily would PhD E: Mm - hmm. Yeah. Professor A: and {disfmarker} and people say it different ways. PhD E: Yep. Grad C: OK. Is this a change from the last batch of {disfmarker} of um forms? Because in the last batch it was spelled out which one you should read. PhD E: Yeah, it was orthographic, so. Professor A: Yes. That's right. It was {disfmarker} it was spelled out, and they decided they wanted to get at more the way people would really say things. Grad C: Oh. OK. Professor A: That's also why they're {disfmarker} they're bunched together in these different groups. So {disfmarker} so it's {disfmarker} Grad C: OK. Professor A: Yeah. So it's {disfmarker} it's {disfmarker} Everything's fine. Grad C: OK. Professor A: OK. Actually, let me just s since {disfmarker} since you brought it up, I was just {disfmarker} it was hard not to be self - conscious about that when it {vocalsound} after we {disfmarker} since we just discussed it. But I realized that {disfmarker} that um {vocalsound} when I'm talking on the phone, certainly, and {disfmarker} and saying these numbers, {vocalsound} I almost always say zero. And uh {disfmarker} cuz {disfmarker} because uh i it's two syllables. It's {disfmarker} it's more likely they'll understand what I said. So that {disfmarker} that {disfmarker} that's the habit I'm in, but some people say" O" and {disfmarker} Grad B: Yeah I normally say" O" cuz it's easier to say. Professor A: Yeah it's shorter. Yeah. So it's {disfmarker} So. {vocalsound} So uh. Grad B:" O" Professor A: Now, don't think about it. Grad B: Oh, no! Professor A: OK. We're done.
Grad B was working with Larry Saul on factorial HMM and doing inference with them. Grad B thought that perhaps modulation spectrum could be used on features and sub-bands to advance this work.
What did Grad B think about time series?