Name: Probability, random variables, and stochastic processes
ISBN: 9780070484689

What Hacker News says

nextosJan 2023

Hubbard & Hubbard, which I referred to in my previous comment, discusses the basics of probability up to the Central Limit Theorem and a bit of Monte Carlo methods. Probability and statistics are really broad. What areas are you interested in?

If you are trying to do inference on large datasets, I think Andrew Gelman's books are superb and will teach you invaluable skills using multi-level Stan models, i.e. generative models and Bayesian inference.

If you want to study basic probability theory and stochastic processes, I really like Elementary Probability Theory by Chung. His more advanced book is really famous, but has a measure-theoretic approach and that's not very practical unless you are interested in developing theories a bit further.

Taleb is really fond of Probability, Random Variables and Stochastic Processes by Papoulis. But I find the typesetting in later editions to be really disorganized and confusing. Take a look. It's a good alternative to the first Chung book.

thread

aalhourJul 2020

Hello HN,

What would be a good alternative text to the "Probability, Random Variables, and Stochastic Processes" book? I couldn't find it for a cheap price in Europe (~90 eur).

Link: https://www.amazon.de/dp/0071226613/

thread

graycatApr 2020

Some quite careful math for Fourier series is in Rudin, Principles of Mathematical Analysis. Fourier series applies to a function on the whole real line that is periodic, that is, repeats exactly once each some number of seconds. For the math, need only one period so in effect throw away the rest of the function and, indeed, really need the function defined only on the interval of some one period.

For some intuition, consider music, especially on a violin. Fourier series applies to a periodic function (wave), and represents the whole wave as sine waves that fit the one period exactly. So, get sine waves at frequency 1, 2, ... that of the period. In music, these waves are called overtones.

Playing with a violin, the overtones are fully real and even important! E.g., get a tuning fork and tune the A string (second from the right as the violinist sees them) to 440 cycles per second (440 Hertz, 440 Hz). Then the D string, the next to the left, is supposed to have frequency 2/3rds that of the A string. So, bow the two strings together and listen for the pitch 880 Hz, that is, 3 times the desired frequency of the D string and twice that of the A string. So are listening to the second overtone of the D string and the first overtone of the A string; are hearing the third Fourier series term of the D string and the second Fourier series term of the A string. Adjust the tuning peg of the D string until don't hear beats. If the D string is at, say, 881 Hz, then will get 1 beat a second -- so this is an accurate method of tuning. Similarly for tuning the E string from the A string and the G string from the D string -- on a violin, the frequencies of adjacent strings are in the ratio of 3:2, that is, a perfect fifth. That's how violinists tune their violin -- which is needed often since violins are just wood and glue and less stable than, say, the cast iron frame of a piano.

For one more, hold a finger lightly against a string at 1/2 the length of the string and hear a note one octave, twice the frequency, higher. That's often done in the music, e.g., playing harmonics. And it's a good way to get the left hand where it belongs at the start of the famous Bach Preludio in E-major that starts on the E half way up the E string. Lightly touch one third of the way up the string and get three times the fundamental frequency, sometimes done in music to give a special tone color. Net, Fourier series, harmonics, and overtones are real everyday for violinists.

E.g., on a piano, hold down a key and then play and release the key one octave lower and notice that the strings of the key held down still vibrate. The key vibrating was stimulated by the first overtone of the key struck and released.

The Fourier integral applies to functions on the whole real line. Very careful math is in Rudin, Real and Complex Analysis.

Yes, Fourier series and integrals can be looked at as all about perpendicular projections of rank 1 as emphasized in Halmos, Finite Dimensional Vector Spaces, written in 1942 when Halmos was an assistant to John von Neumann at the Institute for Advanced Study. That Halmos book is a finite dimensional (linear algebra) introduction to Hilbert space apparently at least partly due to von Neumann. So, right, Fourier theory can be done in Hilbert space.

Fourier integrals and series are very close both intuitively and mathematically, one often an approximation to the other. E.g., if multiply in one (time, frequency) domain, then convolve in the other (frequency, time) domain. E.g., take a function on the whole real line, call it a box, that is 0 everywhere but 1 on, say, [-1,1]. Well the Fourier transform of the box is a wave, roughly a bell curve, that goes to zero quickly away from 0. A convolution is just a moving weighted average, usually a smoothing. Then given a function on the whole real line, regard that line as the time domain and multiply by the box. Now can regard the result as one period, under the box, of a periodic function to which can apply Fourier series. And in the frequency domain, the Fourier transform of the product is the smoothing with the Fourier transform of the box of the Fourier transform of the function and, then, an approximation of Fourier series coefficients of a periodic function with the one period under the box. Nice. That is partly why the fast Fourier transform algorithm is presented as applying both to Fourier series and the Fourier transform.

Mostly Fourier theory is done with an L^2, that is, finite square integral, assumption, but somewhere in my grad school notes I have some of the theory with just an L^1 assumption. Nice notes!

Essentially the Fourier transform of a Gaussian bell curve is a Gaussian bell curve -- if the curve is wide in one domain, then it is narrow in the other.

The uncertainty principle in quantum mechanics is just Plancherel's theorem from Fourier theory.

Can do a lot with Fourier theory just with little pictures such as for that box -- can get a lot of intuition for what is actually correct.

I got all wound up with this Fourier stuff when working on US Navy sonar signal processing.

Then at one point I moved on to power spectral analysis of wave forms, signals, sample paths of stochastic processes, as in

Blackman and Tukey, The Measurement of Power Spectra: From the Point of View of Communications Engineering.

Can get more on the relevant wave forms, signals, stochastic processes from

Athanasuis Papoulis, Probability, Random Variables, and Stochastic Processes, ISBN 07-048448-1.

with more on the math of the relevant stochastic processes in a chapter of

J. L. Doob, Stochastic Processes.

Doob was long a leader in stochastic processes in the US and the professor of Halmos.

At one time a hot area for applications of Fourier theory and the fast Fourier transform was to looking for oil, that is, mapping underground layers, as in

Enders A. Robinson, Multichannel Time Series Analysis with Digital Computer Programs.

Quickly antenna theory depends deeply on Fourier theory so can do beam forming, etc.

Can also see

Ron Bracewell, The Fourier Transform and its Applications.

Of course, one application is to holography. So, that's why can cut a hologram in half and still get the whole image, except with less resolution: The cutting in half is like applying that box, and the resulting Fourier transform is just the same as before except smoothed some by the Fourier transform of the box.

As I recall, in

David R. Brillinger, Time Series Analysis: Data Analysis and Theory, Expanded Edition, ISBN 0-8162-1150-7,

every time-invariant linear system (maybe with some meager additional assumptions) has sine waves as eigenvectors. That it, feed in a sine wave and, then, will get out a sine wave with the same frequency but maybe with amplitude and phase adjusted.

So, in a concert hall, the orchestra plays and up in the cheap seats what hear is the wave filtered by a convolution, that is, with the amplitudes and phases of the Fourier transform of the signal adjusted by the characteristics of the concert hall.

In particular, the usual audio tone controls are essentially just such adjustments of Fourier transform amplitudes and phases.

Since there a lot of systems that are time-invariant and linear or nearly so, there is no shortage of applications of Fourier theory.

thread

mnemonicslothOct 2007

Here are some background and pointers. I'm not a computer vision guy, but these might get you started.

Computer vision is really hard. The computational resources we can spend on the problem are orders of magnitude smaller than what we're trying to emulate in the nature. I think (IANACVS!) something like 35% of the human brain's processing capacity deals with vision, for example. So the discipline is both highly mathematical (to root out shortcuts and shrink the problem) and computationally intense. Also, it's often got some kind of real-time requirements.

Don't let me discourage you though. Since I don't know much about the field, I don't know any of its successes. CV researchers have produced useful systems.

1. Image transform methods: looking up the 2D Fourier transform might be a good start.

2. Variational methods. These are hard to explain as they relate to vision, but if you read the first few chapters of Sussman & Wisdom's Structure & Interpretation of Classical Mechanics, you'll get an idea of what they're talking about.

3. Anything that involves sensors also involves noise. At my school, the first stage of the graduate CV track is a course based on [Papoulis & Pillai]'s Probability, Random Variables, and Stochastic Processes.

4. One of my classmates was involved in object modeling with manifolds. A manifold is basically an n-dimensional shape that follows mostly-Euclidean rules like a sphere, torus, or pretzel. I have no idea what to recommend here, because my curiosity about manifolds and repeated frustration in my efforts to find out more about them ultimately landed me in grad school. Watch out for that.

5. Obviously I know more about software than hardware, but you could definitely do worse than reading up on commonly-used sensors. A midlevel text on CCD cameras would be the way to go here.

6. You can read up on Visual Perception in humans. When I took a class on this as an undergrad, it was a senior-level psych elective. A related field is Psychophysics, which uses methods from physics to study sensory stimuli (e.g. acoustic waves) as they relate to perceived experience. See #s 1 and 3 above.

Hope some of this helps.

thread