Author Topic: AI - Artificial Intelligence  (Read 17041 times)

Liberty

  • Lifetime Member
  • Hero Member
  • *****
  • Posts: 11070
  • twitter.com/libertyRPF
    • twitter.com/libertyRPF
Re: AI - Artificial Intelligence
« Reply #50 on: March 13, 2019, 06:05:20 PM »
AI algorithms actually are just an application of technical analysis, but in a diiferent context.
'History can predict future events'; in tech speak, make the machine calculate all possible correlations in a data set - & it WILL find some that are 'somewhat' predictive (middling R-square values). As it applies these correlations, we call it 'learning'. Of course, the 'machine' is only as 'smart' as the R-square of the correlation, and it's stability in an out-of-sample application; introduce it to a market-discontinuity, and it goes beserk :)

One of the theoretical arguments around HFT is that if your holding period is very small (nano-seconds), almost all your price gain will be attributable to market drift; and we can calculate the amount of that drift, using the Brownian Motion equations. Applied to AI, the more you can apply the Brownian Motion equations to an AI algorithm, the more accurate and stable it becomes.   

All things coming out of the 'investment' silo, and making the jump into other places.

SD

 ???
"Most haystacks don't even have a needle." |  I'm on Twitter  | This podcast episode is a must-listen


Cigarbutt

  • Hero Member
  • *****
  • Posts: 1550
Re: AI - Artificial Intelligence
« Reply #51 on: March 13, 2019, 08:05:57 PM »
An interesting thread. One thing I notice is that the AI algorithms discussed don't appear to try to understand the behaviour of the markets in terms of the trading strategies of the market participants. Instead they try to predict future prices based purely on past price information. In my mind this is surely putting the cart before the horse, in the sense that in the markets, price and price changes are what result FROM the actions of market participants, not the other way round. Yes it is true that the actions of market participants are often driven by price, but it is always historical price information that is taken into consideration, even if it is a few milliseconds ago - the current market price is never precisely known until the order is placed and the trade confirmed.

The price at any moment in time is always the price at which buy and sell volume are exactly matched - if it is not, the price moves up and down in an instant (thanks to high-frequency traders) until it is exactly matched.

So if you could model the behaviour of market participants in terms of what volume they would each add (if they buy) or subtract (if they sell), then you could model the future of price changes.

Now of course this is very difficult because peoples' trading strategies are often complicated, chaotic, and subject to emotional influence, but it occurs to me that many new traders in particular are likely to be using simple trading strategies based on popular technical analysis methods, and similar. If we wanted to model this behaviour using AI we could potentially do this. And if this model was able to do this successfully then we would be in a better position to use AI to go on to model the prices that are more likely to occur as a result of this behaviour.

AI algorithms actually are just an application of technical analysis, but in a diiferent context.
'History can predict future events'; in tech speak, make the machine calculate all possible correlations in a data set - & it WILL find some that are 'somewhat' predictive (middling R-square values). As it applies these correlations, we call it 'learning'. Of course, the 'machine' is only as 'smart' as the R-square of the correlation, and it's stability in an out-of-sample application; introduce it to a market-discontinuity, and it goes beserk :)

One of the theoretical arguments around HFT is that if your holding period is very small (nano-seconds), almost all your price gain will be attributable to market drift; and we can calculate the amount of that drift, using the Brownian Motion equations. Applied to AI, the more you can apply the Brownian Motion equations to an AI algorithm, the more accurate and stable it becomes.   

All things coming out of the 'investment' silo, and making the jump into other places.

SD
...
???
I find the above comments quite interesting.
I've been using voice recognition software for quite some time and the technology relies on deep learning and machine learning, both subsets of artificial intelligence. Through recognition of voice patterns, the software reproduces written text and, over time, gets better at it. But the technology remains quite poor concerning certain aspects that require basic common sense (when I use a new word, a word in a different language, someone else speaks) and the "machine" does not recognize an obvious mistake with very potentially consequential impact on the substance of the underlying message. Proofreading has become markedly different as the software (even if amazingly efficient at certain tasks) can produce very stupid results.

neil9327's point, I think, was that we would hope to integrate and or understand the underlying "behavior" that led to the subsequent price action in order to improve prediction capabilities. The best short-term predictive ability of where a stock will go is what it did in the short-term past and this has been captured by simple linear regression models assuming markets function linearly most of the times (with some predictable variation) and this is where correlation coefficients and R-squared values come into play.

The idea (and the hope at this point) of machine learning and higher artificial neural networks for better prediction capabilities relies on improved pattern classification and ability to recognize patterns on its own in order to determine non-linear extrapolations.

In a way, this is nothing new as Thomas Bayes described the foundation of machine learning in 1763. Pattern recognition can be improved but the underlying principles that rest on past behavior can lead one astray (such as when using the VaR concepts) especially when transitions occur between calm and chaos or vice-versa. Artificial intelligence will need to integrate behavioral aspects and IMHO we're not quite there yet on many levels.

One of the biggest risks may be missing the forest for the trees (bigger picture, perspective etc) because the complexity of the model and the huge amount of data used may result in an illusory sense of precision.

I would say pattern recognition has value but is only a starting Brownian point for deep and independent thinking.
Potential bias: "Investment is most intelligent when it is most businesslike."

Jurgis

  • Hero Member
  • *****
  • Posts: 4404
    • Porfolio
Re: AI - Artificial Intelligence
« Reply #52 on: March 14, 2019, 06:55:07 AM »
"Before you can be rich, you must be poor." - Nef Anyo
--------------------------------------------------------------------
"American History X", "Milk", "The Insider", "Dirty Money", "LBJ"

Jurgis

  • Hero Member
  • *****
  • Posts: 4404
    • Porfolio
"Before you can be rich, you must be poor." - Nef Anyo
--------------------------------------------------------------------
"American History X", "Milk", "The Insider", "Dirty Money", "LBJ"

SharperDingaan

  • Lifetime Member
  • Hero Member
  • *****
  • Posts: 3108
Re: AI - Artificial Intelligence
« Reply #54 on: March 16, 2019, 05:57:35 AM »
An interesting thread. One thing I notice is that the AI algorithms discussed don't appear to try to understand the behaviour of the markets in terms of the trading strategies of the market participants. Instead they try to predict future prices based purely on past price information. In my mind this is surely putting the cart before the horse, in the sense that in the markets, price and price changes are what result FROM the actions of market participants, not the other way round. Yes it is true that the actions of market participants are often driven by price, but it is always historical price information that is taken into consideration, even if it is a few milliseconds ago - the current market price is never precisely known until the order is placed and the trade confirmed.

The price at any moment in time is always the price at which buy and sell volume are exactly matched - if it is not, the price moves up and down in an instant (thanks to high-frequency traders) until it is exactly matched.

So if you could model the behaviour of market participants in terms of what volume they would each add (if they buy) or subtract (if they sell), then you could model the future of price changes.

Now of course this is very difficult because peoples' trading strategies are often complicated, chaotic, and subject to emotional influence, but it occurs to me that many new traders in particular are likely to be using simple trading strategies based on popular technical analysis methods, and similar. If we wanted to model this behaviour using AI we could potentially do this. And if this model was able to do this successfully then we would be in a better position to use AI to go on to model the prices that are more likely to occur as a result of this behaviour.

AI algorithms actually are just an application of technical analysis, but in a diiferent context.
'History can predict future events'; in tech speak, make the machine calculate all possible correlations in a data set - & it WILL find some that are 'somewhat' predictive (middling R-square values). As it applies these correlations, we call it 'learning'. Of course, the 'machine' is only as 'smart' as the R-square of the correlation, and it's stability in an out-of-sample application; introduce it to a market-discontinuity, and it goes beserk :)

One of the theoretical arguments around HFT is that if your holding period is very small (nano-seconds), almost all your price gain will be attributable to market drift; and we can calculate the amount of that drift, using the Brownian Motion equations. Applied to AI, the more you can apply the Brownian Motion equations to an AI algorithm, the more accurate and stable it becomes.   

All things coming out of the 'investment' silo, and making the jump into other places.

SD
...
???
I find the above comments quite interesting.
I've been using voice recognition software for quite some time and the technology relies on deep learning and machine learning, both subsets of artificial intelligence. Through recognition of voice patterns, the software reproduces written text and, over time, gets better at it. But the technology remains quite poor concerning certain aspects that require basic common sense (when I use a new word, a word in a different language, someone else speaks) and the "machine" does not recognize an obvious mistake with very potentially consequential impact on the substance of the underlying message. Proofreading has become markedly different as the software (even if amazingly efficient at certain tasks) can produce very stupid results.

neil9327's point, I think, was that we would hope to integrate and or understand the underlying "behavior" that led to the subsequent price action in order to improve prediction capabilities. The best short-term predictive ability of where a stock will go is what it did in the short-term past and this has been captured by simple linear regression models assuming markets function linearly most of the times (with some predictable variation) and this is where correlation coefficients and R-squared values come into play.

The idea (and the hope at this point) of machine learning and higher artificial neural networks for better prediction capabilities relies on improved pattern classification and ability to recognize patterns on its own in order to determine non-linear extrapolations.

In a way, this is nothing new as Thomas Bayes described the foundation of machine learning in 1763. Pattern recognition can be improved but the underlying principles that rest on past behavior can lead one astray (such as when using the VaR concepts) especially when transitions occur between calm and chaos or vice-versa. Artificial intelligence will need to integrate behavioral aspects and IMHO we're not quite there yet on many levels.

One of the biggest risks may be missing the forest for the trees (bigger picture, perspective etc) because the complexity of the model and the huge amount of data used may result in an illusory sense of precision.

I would say pattern recognition has value but is only a starting Brownian point for deep and independent thinking.
Potential bias: "Investment is most intelligent when it is most businesslike."

Good points.

The other issue with AI is that while development is happening in many different places, there's a lot of contagion as the many groups jump off each others innovations. While inherent to the scientific discovery, and agile project management, process; it produces 'dogma', along with discovery.

"We invented it, this is how you do it, don't presume to tell me otherwise".
At one time, we were also 'sure', that the earth was at the centre of the universe.

For AI to work commercially, it needs you and I to permit it 'free' access to large amounts of 'our' transaction data.
Whether at the granular, or meta-data level; that data is an asset - and you will be chaged to use it.
All learning has an ongoing 'tuition cost'.

To avoid spurious results, the data also has to be be complete - and accurate. Ever looked at historic data? It's full of inaccuracies,
Ever looked at blockchain data? Every transaction record is perfect and 2nd party verified - but you dont get it unless the Oracle makes it available to the public (distrubted/private ledgers). Oracles are the toll-booth trolls, and you WILL pay them to access their 'golden data'.

Blockchain/AI are opposit sides of the same coin ... but blockchain is the 'control' side of the coin.
Something that AI has been reluctant to recognize.

SD
« Last Edit: March 16, 2019, 06:02:05 AM by SharperDingaan »

Jurgis

  • Hero Member
  • *****
  • Posts: 4404
    • Porfolio
Re: AI - Artificial Intelligence
« Reply #55 on: April 17, 2019, 03:08:01 AM »
http://fortune.com/longform/ai-drugs-pharma-pharnext-cmt/ - somewhat fluff piece, but covers the issues of going from gene-disease mapping to actual drugs that may treat a disease.
"Before you can be rich, you must be poor." - Nef Anyo
--------------------------------------------------------------------
"American History X", "Milk", "The Insider", "Dirty Money", "LBJ"

Jurgis

  • Hero Member
  • *****
  • Posts: 4404
    • Porfolio
"Before you can be rich, you must be poor." - Nef Anyo
--------------------------------------------------------------------
"American History X", "Milk", "The Insider", "Dirty Money", "LBJ"

rkbabang

  • Lifetime Member
  • Hero Member
  • *****
  • Posts: 4229
Re: AI - Artificial Intelligence
« Reply #57 on: April 19, 2019, 10:36:57 AM »
‘You Can’t Take My Door’, A Country Song Created by a Neural Network That Studied a Catalog of Country Hits
https://laughingsquid.com/country-song-created-by-neural-network/

rkbabang

  • Lifetime Member
  • Hero Member
  • *****
  • Posts: 4229
Re: AI - Artificial Intelligence
« Reply #58 on: April 19, 2019, 11:11:44 AM »
And even better.  I've found this.  A deep learning death metal generator.  It constantly streams deep learning generated death metal in real time as it generates it 24/7.  It isn't bad. Something you can just keep on your headphones as you work.

https://www.youtube.com/watch?v=CNNmBtNcccE

cherzeca

  • Hero Member
  • *****
  • Posts: 1876
Re: AI - Artificial Intelligence
« Reply #59 on: April 19, 2019, 11:12:53 AM »
in terms of AI used in equity investing, I am thinking that i) it is likely happening successfully now, as it seems Simon must be using some form of AI (or does Simon focus on other markets than equities?), and ii) AI pattern recognition is becoming profitably used in medical diagnostics (now tumor detection by AI is better than by a radiologist using eyes alone).

in a sense, the yes/no tumor detection decision is binary in the same sense as the gain/loss investment decision, but I wonder if the inputs are too multivariate in the investment context.  whether with go or chess, there are finite rules, although near infinite permutations. I wonder if there are infinite rules (let alone permutations) in equity investing...
« Last Edit: April 19, 2019, 11:15:40 AM by cherzeca »