Welcome, Guest. Please Login
Algorithmic Trading Forum
 
  HomeHelpSearchLogin  
 
Pages: 1
Send Topic Print
Ok, guys, help with a real project architecture! (Read 14772 times)
qroach
Full Member
***
Offline



Posts: 102
London
Gender: male
Ok, guys, help with a real project architecture!
09/02/08 at 15:23:36
 
Trying to enlighten the forum again...

This might seem stupid and a very dated idea. It is certainly below the level of at least some forum members. But it can be a good starting point for many.

There is a rather old concept of relative strength (RS) for equities. On any given day, If we take all stock dayly returns and order them in descending order we will be able to rank stocks from the highest return on that day to the lowest. Instead of considering nominal ranks we can split the whole ordered list into percentiles and, say, select the top 99 percentile (i.e. 1% of the highest returns on that day). That number (99) will be RS for the stock.

Now, for each stock, instead of having returns time series we can consider instead RS time series. Presumably, stocks don't jump from 99 to 1 on day-to day basis but rather have some pattern, which can be investigated.

So, one task would be to download all stocks history and transform it into RS history.

Next task would be to measure profitability of a simple portfolio that is distributed amongst top 1-2-3% and is rebalanced periodically (daily)

Another task is to take into account survivorship bias and measure its influence on the return on the above mentioned portfolio.

I am asking the forum members to contribute their ideas on how to implement a system that can do these 3 simple tasks. I only need software architecture ideas (hence, the location of the thread), namely:
1) where to take data from,
2) how to transform it
3) how to backtest the portfolio given the data obtained in 1) and 2)

I am not very interested in turn-key commercial solutions. I only want to build it myself from several pieces because it will allow me to do a research on the obtained data in my own unique way.

Comments like "it is all not very gifted" are welcome.
Back to top
 
 
  IP Logged
Co0olCat
Forum Moderator
*****
Offline



Posts: 128

Gender: male
Re: Ok, guys, help with a real project architecture!
Reply #1 - 09/02/08 at 17:24:55
 
Off-shelf solution is to use PosgreSQL. You can write your own functions there. Data is from Yahoo or Google. In R you can find functions to download daily data. These functions create URL request and store CSV files to the disk. These files you can feed directly to the DB. Yahoo provides adjusted close prices to take into accounts splits and dividends.

After that you can perform whatever studies to investigate profitability and associated risk.
Back to top
 
 

"Yesterday is history. Tomorrow is a mystery. But today is a gift, and that is why it's called the present."
WWW timuryusupov   IP Logged
qroach
Full Member
***
Offline



Posts: 102
London
Gender: male
Re: Ok, guys, help with a real project architecture!
Reply #2 - 09/02/08 at 17:42:24
 
Ok, thanks!

Haven't discovered yet how to download quotes from Yahoo in R - should be very handy.

Why not MySQL? What is so special about Postgress? Never used it though....

The last task is still a problem - how to find/keep track of dead companies to measure the survovorship bias? Or should it normally be not too big - It is still a mistery to me.
Back to top
 
 
  IP Logged
kirppu
Newbie
*
Offline

I like this forum!

Posts: 15
Dublin
Gender: male
Re: Ok, guys, help with a real project architecture!
Reply #3 - 09/03/08 at 09:45:51
 
Here is a sample java code to automatically download quotes from yahoo and convert it into a postgreSQL insert statement.

Back to top
 
  IP Logged
kirppu
Newbie
*
Offline

I like this forum!

Posts: 15
Dublin
Gender: male
Re: Ok, guys, help with a real project architecture!
Reply #4 - 09/03/08 at 09:50:33
 
To automatically download and insert yahoo quotes in a postgreSQL table, use the previous java code and the following script:

#!/bin/bash

# 1) write the password for psql

export PGPASSWORD=xyz # edit you passoword

# 2) name of all yahoo symbols you want to download

a=( AAPL
ABB
^OMX
^RUT
^SPMIB
^SSMI)


COUNTER=0
while [ $COUNTER -lt ${#array
  • } ]; do
      symbol=${a[$COUNTER]}
      file=$symbol".csv"

      #. 3. create an INSERT for each symbol
      java Insert $file $symbol

      #. 4. insert the timeseries in the database taq
      psql taq -f $file &>0

      # 5. remove the file
      rm $file
      let COUNTER=$COUNTER+1
    done

  • Back to top
     
     
      IP Logged
    kirppu
    Newbie
    *
    Offline

    I like this forum!

    Posts: 15
    Dublin
    Gender: male
    Re: Ok, guys, help with a real project architecture!
    Reply #5 - 09/03/08 at 09:52:02
     
    Can somebody send us the equivalent in R?
    Back to top
     
     
      IP Logged
    Co0olCat
    Forum Moderator
    *****
    Offline



    Posts: 128

    Gender: male
    Re: Ok, guys, help with a real project architecture!
    Reply #6 - 09/03/08 at 10:06:51
     
    To get data in R there is Quantmod package: http://www.quantmod.com

    Check getSymbols.oanda(), getSymbols.google(), getSymbols.yahoo()

    As for SQL you can use RODBC: http://cran.r-project.org/web/packages/RODBC/index.html

    Note: To avoid performance issue I would suggest to import data to a DB through CSV file. Use PostgreSQL command COPY.
    Back to top
     
     

    "Yesterday is history. Tomorrow is a mystery. But today is a gift, and that is why it's called the present."
    WWW timuryusupov   IP Logged
    qroach
    Full Member
    ***
    Offline



    Posts: 102
    London
    Gender: male
    Re: Ok, guys, help with a real project architecture!
    Reply #7 - 09/04/08 at 19:10:14
     
    I would also download everthing into cvs file and then do bulk import into SQL server of choice.

    I would prefer R-solution to Java+bash just to keep it simple at the begining.

    Quantmod is good in a sense that it allows to limit the development scope - it can download from Yahoo through R, then save it in MySql or RODBC in R and retrieve it from MySQL or RODBC in R. Great!

    But thanks to either of you!
    Back to top
     
     
      IP Logged
    qroach
    Full Member
    ***
    Offline



    Posts: 102
    London
    Gender: male
    Re: Ok, guys, help with a real project architecture!
    Reply #8 - 09/06/08 at 20:01:18
     
    Ok, this is my first attempt to download the data:
    1. I went to finviz.com screener, selected "Average volume between 500k and 1m" got 686 stocks as a result, which I downloaded as "finviz.csv" file

    2. This is the R script (using quantmod):
    Code:
    require("quantmod")
    allstocks = read.csv("C:/data/stocks/finviz.csv", stringsAsFactors=F)
    for(i in 1:length(allstocks$Ticker)) {
      t = as.character(allstocks$Ticker[i])
      stock = as.data.frame(getSymbols(t, src="yahoo", auto.assign=F))
      names(stock) = c("Open", "High", "Low", "Close", "Volume", "Adjusted")
      fname = paste(t, ".csv", sep="")
      fname = paste("C:/data/stocks/", fname, sep="")
      write.csv(stock, fname)
    }
     
    
    


    It took about 10 min to download the whole lot. Now I have got plenty of csv-files. And thinking of what to do next with them. To process, to SQL import them....
    Back to top
     
     
      IP Logged
    Pages: 1
    Send Topic Print