in All availableThis forumThis topic
 Prediction algorithms
Author Message
richk449
Guru

Joined: 24 Oct 2003
Posts: 345

 Posted: Tue Apr 10, 2012 2:49 am    Post subject: Prediction algorithms Anyone know anything about prediction algorithms? I have some data, which consists of design information and results. In simplified form, the design data for each entry consists of 12 numbers between 0 and 1, and the results data is a single value, the higher the better. I want to use the design data to predict a good place to search for results. Depending on the specific problem, I have somewhere between 500 and 5000 entries to use. I naively tried to implement a singular value decomposition based dimensionality reduction algorithm, like the one described here: http://sifter.org/~simon/journal/20061211.html. It works great on a toy problem, but when I try to apply it to a real problem, I quickly run into trouble. The netflix algorithm is set up for essentially two dimensional data - people on one axis, and movies on another axis. My data, on the other hand, is 12 dimensional. I start by discretizing my continuous design data into bins. If I use a fairly meager 10 bins between 0 and 1, I already have 10^12 in a full array. To use the Netflix algorithm, I flatten the 12 dimensional array into a two dimensional array of 10^6 in each dimension. If I work with a reduced dimension of, say, 10, I am left trying to train a 10^7 array with ~10^4 data points. That doesn't even come close to working. So after implementing all of this, I now understand enough about the algorithm I am using to realize it is totally inappropriate. However, I don't really want to go through the same experience with 10 more algorithms until I find one that kinda works. So does anyone have any knowledge of prediction algorithms?
aCOSwt

Joined: 19 Oct 2007
Posts: 2041
Location: Between the keyboard and the chair

Posted: Tue Apr 10, 2012 9:50 am    Post subject: Re: Prediction algorithms

 richk449 wrote: to predict a good place to search for results... I naively tried to implement a ... reduction algorithm... So after implementing all of this, I now understand enough about the algorithm I am using to realize it is totally inappropriate. However, I don't really want to go through the same experience with 10 more algorithms until I find one that kinda works.

We... have all experienced this... exactly !

What are you looking for ? I mean a theory or ready to use libraries ?

If you look for ready to use libraries then I warmly recommend : The Shogun Toolbox
You might find interesting to read the well documented examples first and see if one particularily looks like what you want to achieve.
BTW, first consider those listed under octave-modular.
If you install as follows then, these will be the ones that match your installation.
If you prefer python then look in python-modular. I did not test the python bindings.

You can find an ebuild on zugaina.
Opt for 1.1.0

Nota 1 about the ebuild : You can safely change the original
 Code: PYTHON_DEPEND="python? 2:2.6"
against
 Code: PYTHON_DEPEND="python? 2:2.7"

Nota 2 : It builds fine with R and python as backends but... I did not manage to use shogun libraries with R. There seems to be some conflict in namespaces, I had no time to investigate.
So, I chose Octave as a backend.
 Code: sci-libs/shogun-1.1.0::PrimaPratica_ovl  USE="-R -java -lua -mono octave -python -ruby" 0 kB

It works perfectly but the build you obtain on zugaina is broken when building +octave alone.

If you want to build shogun like I did then you need to add a dependency on swig which was forgotten :
 Code: octave? ( sci-mathematics/octave                   dev-lang/swig )

You need very last swig, that is : 2.0.4-r1

Enjoy !

BTW... are you having some competitive examination in France next week ?
_________________
In theory there are no differences between theory and practice. In practice, there are.
Don't try to understand my posts. Immanuel Kant never did, he thinks that only music and laughter do not have to mean anything.
wildhorse
Tux's lil' helper

Joined: 16 Mar 2006
Posts: 148

 Posted: Tue Apr 10, 2012 11:08 am    Post subject: Looks like a good use case for classification using the principle components analysis (PCA). Next year's winner may use the independent component analysis. Send me 50% of the award.
pigeon768
l33t

Joined: 02 Jan 2006
Posts: 667

 Posted: Tue Apr 10, 2012 12:03 pm    Post subject: http://en.wikipedia.org/wiki/Hidden_Markov_model ?_________________My political bias.
wildhorse
Tux's lil' helper

Joined: 16 Mar 2006
Posts: 148

 Posted: Tue Apr 10, 2012 12:41 pm    Post subject: Lots of fancy stuff is summarised at Support Vector Machines. Another very simple ansatz would be the use of a back-propagation neural network. Search the web for "clustering backpropagation". I have actually used sci-mathematics/fann for that purpose myself. Note: the PCA does address exactly the complications which make it so hard for richk449 to become just rich. I am quite familiar with the PCA and the SVD, if I may add. Stock market anyone? Please!
richk449
Guru

Joined: 24 Oct 2003
Posts: 345

Posted: Tue Apr 10, 2012 2:56 pm    Post subject: Re: Prediction algorithms

 aCOSwt wrote: What are you looking for ? I mean a theory or ready to use libraries ?

Either one, but if it is a theory, it needs to be a simple one, and if it is a library, it needs to be a simple one.

Unfortunately, I typically find in these types of situations that in order to use the ready-made libraries, you need to be an expert in the field already. And I am definitely not that.

 Quote: If you look for ready to use libraries then I warmly recommend : The Shogun Toolbox You might find interesting to read the well documented examples first and see if one particularily looks like what you want to achieve.

Thanks for the recommendation. There are a lot of software packages out there that claim to implement various types of machine learning, so I was really hoping to get some recommendations on what others prefer. This one looks pretty good, and they do provide examples, which is exactly what I need.

 Quote: BTW... are you having some competitive examination in France next week ?

No, but I wish I had an excuse to go to France next week.

Last edited by richk449 on Tue Apr 10, 2012 3:03 pm; edited 1 time in total
richk449
Guru

Joined: 24 Oct 2003
Posts: 345

Posted: Tue Apr 10, 2012 3:02 pm    Post subject:

 wildhorse wrote: Lots of fancy stuff is summarised at Support Vector Machines. Another very simple ansatz would be the use of a back-propagation neural network. Search the web for "clustering backpropagation". I have actually used sci-mathematics/fann for that purpose myself. Looks like a good use case for classification using the principle components analysis (PCA). Note: the PCA does address exactly the complications which make it so hard for richk449 to become just rich. I am quite familiar with the PCA and the SVD, if I may add. Stock market anyone? Please!

Thanks for the suggestions. I will look into them.

There used to be a time when I dreamed about making money in the stock market with better match. Nowdays though, there is no way an individual could compete with the big firms, with their teams of quants and supercomputers and fat pipes straight to the exchanges. Perhaps if you genuinely had a better idea, you could compete for a while (until they figured out your idea), but it is not easy to have better ideas than teams of well educated people who are heavily incentivized to have the best ideas.
rtomek
Apprentice

Joined: 05 Jan 2007
Posts: 210
Location: Chicago

 Posted: Tue Apr 10, 2012 4:36 pm    Post subject: Well there is this if you want to get started: http://videolectures.net/mlss09us_chicago/ You probably need a good background in statistics for it. And then if you believe in the efficient market hypothesis, you'll realize that it's way more difficult to make money in the stock market than just having some 'algorithm'. Money is made buy buying the rumor and selling the news, and having an above-average success rate on predicting what rumors are true (or you can make money by cheating other people out of money and try not to get caught doing it).
 Display posts from previous: All Posts1 Day7 Days2 Weeks1 Month3 Months6 Months1 Year Oldest FirstNewest First
 All times are GMT Page 1 of 1

 Jump to: Select a forum Assistance----------------News & AnnouncementsFrequently Asked QuestionsInstalling GentooMultimediaDesktop EnvironmentsNetworking & SecurityKernel & HardwarePortage & ProgrammingGamers & PlayersOther Things GentooUnsupported Software Discussion & Documentation----------------Documentation, Tips & TricksGentoo ChatGentoo Forums FeedbackOff the WallDuplicate Threads International Gentoo Users----------------中文 (Chinese)DutchFinnishFrenchDeutsches Forum (German)  Diskussionsforum  Deutsche DokumentationGreekForum italiano (Italian)  Forum di discussione italiano  Risorse italiane (documentazione e tools)Polskie forum (Polish)  Instalacja i sprzęt  Polish OTWPortuguese  Documentação, Ferramentas e DicasRussianScandinavianSpanishOther Languages Architectures & Platforms----------------Gentoo on AMD64Gentoo on PPCGentoo on SparcGentoo on Alternative ArchitecturesGentoo for Mac OS X (Portage for Mac OS X)
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum