Prediction algorithms

richk449 · Guru Joined: 24 Oct 2003 Posts: 345

Anyone know anything about prediction algorithms?

I have some data, which consists of design information and results. In simplified form, the design data for each entry consists of 12 numbers between 0 and 1, and the results data is a single value, the higher the better. I want to use the design data to predict a good place to search for results. Depending on the specific problem, I have somewhere between 500 and 5000 entries to use.

I naively tried to implement a singular value decomposition based dimensionality reduction algorithm, like the one described here: http://sifter.org/~simon/journal/20061211.html. It works great on a toy problem, but when I try to apply it to a real problem, I quickly run into trouble. The netflix algorithm is set up for essentially two dimensional data - people on one axis, and movies on another axis.

My data, on the other hand, is 12 dimensional. I start by discretizing my continuous design data into bins. If I use a fairly meager 10 bins between 0 and 1, I already have 10^12 in a full array. To use the Netflix algorithm, I flatten the 12 dimensional array into a two dimensional array of 10^6 in each dimension.

If I work with a reduced dimension of, say, 10, I am left trying to train a 10^7 array with ~10^4 data points. That doesn't even come close to working.

So after implementing all of this, I now understand enough about the algorithm I am using to realize it is totally inappropriate. However, I don't really want to go through the same experience with 10 more algorithms until I find one that kinda works.

So does anyone have any knowledge of prediction algorithms?

aCOSwt · Posted: Tue Apr 10, 2012 9:50 am Post subject: Re: Prediction algorithms

wildhorse · Posted: Tue Apr 10, 2012 11:08 am Post subject:

Looks like a good use case for classification using the principle components analysis (PCA). Next year's winner may use the independent component analysis.
Send me 50% of the award.

pigeon768 · l33t Joined: 02 Jan 2006 Posts: 667

http://en.wikipedia.org/wiki/Hidden_Markov_model ?
_________________
My political bias.

wildhorse · Posted: Tue Apr 10, 2012 12:41 pm Post subject:

Lots of fancy stuff is summarised at Support Vector Machines.

Another very simple ansatz would be the use of a back-propagation neural network. Search the web for "clustering backpropagation". I have actually used sci-mathematics/fann for that purpose myself.

Note: the PCA does address exactly the complications which make it so hard for richk449 to become just rich. I am quite familiar with the PCA and the SVD, if I may add.
Stock market anyone? Please! :lol:

richk449 · Guru Joined: 24 Oct 2003 Posts: 345

richk449 · Guru Joined: 24 Oct 2003 Posts: 345

rtomek · Posted: Tue Apr 10, 2012 4:36 pm Post subject:

Well there is this if you want to get started: http://videolectures.net/mlss09us_chicago/

You probably need a good background in statistics for it. And then if you believe in the efficient market hypothesis, you'll realize that it's way more difficult to make money in the stock market than just having some 'algorithm'. Money is made buy buying the rumor and selling the news, and having an above-average success rate on predicting what rumors are true (or you can make money by cheating other people out of money and try not to get caught doing it).