Stock Market Analysis

Aug 06, 2019

Stock Market Data And Analysis In Python

In this article, you will learn to get the stock market data such as price, volume and fundamental data using python packages and how to analyze it.

In backtesting your strategies or analyzing the performance, one of the first hurdles faced is getting the right stock market data and in the right format, isn't it? Don't worry.

After reading this, you will be able to:

• Fetch the open, high, low, close, and volume data.

• Get data at a custom frequency such as 1 minute, 7 minutes or 2 hours

• Perform analysis of your portfolio

• Get the earnings data, balance sheet data, cash flow statements and various key ratios such as price to earnings (PE) and price to book value (PB)

• Get the futures and options data for Indian stock market

Generally, web sources are quite unstable and therefore, you will learn to get the stock market data from multiple web sources. For easy navigation, this article is divided as below.

1. Price Volume Daily Data

2. Intraday Data

3. Fundamental Data

4. Futures and Options Data

5. Visualization and Analysis

Price Volume Daily Data

Yahoo Finance

One of the first sources from which you can get daily price-volume stock market data is Yahoo finance. You can use pandas_datareader or yfinance module to get the data.

In [ ]:

!pip install pandas_datareader==0.7.0

In [22]:

# Import pandas datareader
import pandas_datareader
pandas_datareader.__version__

Out[22]:

'0.7.0'

In [7]:

# Yahoo recently has become an unstable data source.
# If it gives an error, you may run the cell again, or try yfinance
import pandas as pd
from pandas_datareader import data
# Set the start and end date
start_date = '1990-01-01'
end_date = '2019-02-01'
# Set the ticker
ticker = 'AMZN'
# Get the data
data = data.get_data_yahoo(ticker, start_date, end_date)
data.head()

Out[7]:

Date	High	Low	Open	Close	Volume	Adj Close
1997-05-15	2.500000	1.927083	2.437500	1.958333	72156000.0	1.958333
1997-05-16	1.979167	1.708333	1.968750	1.729167	14700000.0	1.729167
1997-05-19	1.770833	1.625000	1.760417	1.708333	6106800.0	1.708333
1997-05-20	1.750000	1.635417	1.729167	1.635417	5467200.0	1.635417
1997-05-21	1.645833	1.375000	1.635417	1.427083	18853200.0	1.427083

To visualize the adjusted close price data, you can use the matplotlib library and plot method as shown below.

In [9]:

import matplotlib.pyplot as plt
%matplotlib inline
data['Adj Close'].plot()
plt.show()

Let us improve the plot by resizing, giving appropriate labels and adding grid lines for better readability.

In [10]:

# Plot the adjusted close price
data['Adj Close'].plot(figsize=(10, 7))
# Define the label for the title of the figure
plt.title("Adjusted Close Price of %s" % ticker, fontsize=16)
# Define the labels for x-axis and y-axis
plt.ylabel('Price', fontsize=14)
plt.xlabel('Year', fontsize=14)
# Plot the grid lines
plt.grid(which="major", color='k', linestyle='-.', linewidth=0.5)
# Show the plot
plt.show()

Advantages

1. Adjusted close price stock market data is available

2. Most recent stock market data is available

3. Doesn't require API key to fetch the stock market data

Disadvantages

1. It is not a stable source to fetch the stock market data

If the stock market data fetching fails from yahoo finance using the pandas_datareaderthen you can use yfinance package to fetch the data.

Quandl

Quandl has many data sources to get different types of data. However, some are free and some are paid. Wiki is the free data source of Quandl to get the data of the end of the day prices of 3000+ US equities. It is curated by Quandl community and also provides information about the dividends and split. To get the stock market data, you need to first install the quandl module if it is not already installed using the pip command as shown below.

In [ ]:

!pip install quandl

You need to get your own API Key from quandl to get the stock market data using the below code. If you are facing issue in getting the API key then you can refer to this link.

After you get your key, assign the variable QUANDLAPIKEYQUANDLAPIKEY with that key. Then set the start date, end date and the ticker of the asset whose stock market data you want to fetch. The quandl get method takes this stock market data as input and returns the open, high, low, close, volume, adjusted values and other information.

In [1]:

# Import the quandl
import quandl
# To get your API key, sign up for a free Quandl account.
# Then, you can find your API key on Quandl account settings page.
QUANDL_API_KEY = 'REPLACE-THIS-TEXT-WITH-A-REAL-API-KEY'
# This is to prompt you to change the Quandl Key
if QUANDL_API_KEY == 'REPLACE-THIS-TEXT-WITH-A-REAL-API-KEY':
 raise Exception("Please provide a valid Quandl API key!")
# Set the start and end date
start_date = '1990-01-01'
end_date = '2018-03-01'
# Set the ticker name
ticker = 'AMZN'
# Feth the data
data = quandl.get('WIKI/'+ticker, start_date=start_date,
 end_date=end_date, api_key=QUANDL_API_KEY)
# Print the first 5 rows of the dataframe
data.head()

Out[1]:

Date	Open	High	Low	Close	Volume	Ex-Dividend	Split Ratio	Adj. Open	Adj. High	Adj. Low	Adj. Close	Adj. Volume
1997-05-16	22.38	23.75	20.50	20.75	1225000.0	0.0	1.0	1.865000	1.979167	1.708333	1.729167	14700000.0
1997-05-19	20.50	21.25	19.50	20.50	508900.0	0.0	1.0	1.708333	1.770833	1.625000	1.708333	6106800.0
1997-05-20	20.75	21.00	19.63	19.63	455600.0	0.0	1.0	1.729167	1.750000	1.635833	1.635833	5467200.0
1997-05-21	19.25	19.75	16.50	17.13	1571100.0	0.0	1.0	1.604167	1.645833	1.375000	1.427500	18853200.0
1997-05-22	17.25	17.38	15.75	16.75	981400.0	0.0	1.0	1.437500	1.448333	1.312500	1.395833	11776800.0

In [3]:

# Define the figure size for the plot
plt.figure(figsize=(10, 7))
# Plot the adjusted close price
data['Adj. Close'].plot()
# Define the label for the title of the figure
plt.title("Adjusted Close Price of %s" % ticker, fontsize=16)
# Define the labels for x-axis and y-axis
plt.ylabel('Price', fontsize=14)
plt.xlabel('Year', fontsize=14)
# Plot the grid lines
plt.grid(which="major", color='k', linestyle='-.', linewidth=0.5)
plt.show()

Get stock market data for multiple tickers

To get the stock market data of multiple stock tickers, you can create a list of tickers and call the quandl get method for each stock ticker.[1] For simplicity, I have created a dataframe data to store the adjusted close price of the stocks.

In [4]:

# Define the ticker list
import pandas as pd
tickers_list = ['AAPL', 'IBM', 'MSFT', 'WMT']
# Import pandas
data = pd.DataFrame(columns=tickers_list)
# Feth the data
for ticker in tickers_list:
 data[ticker] = quandl.get('WIKI/' + ticker, start_date=start_date,
 end_date=end_date, api_key=QUANDL_API_KEY)['Adj. Close']
# Print first 5 rows of the data
data.head()

Out[4]:

Date	AAPL	IBM	MSFT	WMT
1990-01-02	1.118093	14.138144	0.410278	4.054211
1990-01-03	1.125597	14.263656	0.412590	4.054211
1990-01-04	1.129499	14.426678	0.424702	4.033561
1990-01-05	1.133101	14.390611	0.414300	3.990541
1990-01-08	1.140605	14.480057	0.420680	4.043886

In [5]:

# Plot all the close prices
data.plot(figsize=(10, 7))
# Show the legend
plt.legend()
# Define the label for the title of the figure
plt.title("Adjusted Close Price", fontsize=16)
# Define the labels for x-axis and y-axis
plt.ylabel('Price', fontsize=14)
plt.xlabel('Year', fontsize=14)
# Plot the grid lines
plt.grid(which="major", color='k', linestyle='-.', linewidth=0.5)
plt.show()

Advantages

1. It is free of cost

2. Has split and dividend-adjusted stock market data

Disadvantages

1. Only available till 27-March-2018

Intraday Data

Alpha Vantage

Alpha vantage is used to get the minute level stock market data. You need to signup on alpha vantage to get the free API key.

In [ ]:

# Install the alpha_vantage if not already installed
!pip install alpha_vantage

Assign the ALPHA_VANTAGE_API_KEY, with your API Key in the below code.

In [12]:

# Import TimeSeries class
from alpha_vantage.timeseries import TimeSeries
ALPHA_VANTAGE_API_KEY = 'REPLACE-THIS-TEXT-WITH-A-REAL-API-KEY'
# This is to prompt you to change the ALPHA_VANTAGE Key
if ALPHA_VANTAGE_API_KEY == 'REPLACE-THIS-TEXT-WITH-A-REAL-API-KEY':
 raise Exception("Please provide a valid Alpha Vantage API key!")
# Initialize the TimeSeries class with key and output format
ts = TimeSeries(key=ALPHA_VANTAGE_API_KEY, output_format='pandas')
# Get pandas dataframe with the intraday data and information of the data
intraday_data, data_info = ts.get_intraday(
 'GOOGL', outputsize='full', interval='1min')
# Print the information of the data
data_info

Out[12]:

{'1. Information': 'Intraday (1min) open, high, low, close prices and volume',
 '2. Symbol': 'GOOGL',
 '3. Last Refreshed': '2019-08-01 16:00:00',
 '4. Interval': '1min',
 '5. Output Size': 'Full size',
 '6. Time Zone': 'US/Eastern'}

This gives information about the stock market data which is returned. The information includes the type of data returned such as open, high, low and close, the symbol or ticker of the stock, last refresh time of the data, frequency of the stock market data and the time zone.

In [13]:

# Print the intraday data
intraday_data.head()

Out[13]:

Date	Open	High	Low	Close	Volume
2019-07-26 09:31:00	1228.2300	1232.49	1228.0000	1230.7898	407037.0
2019-07-26 09:32:00	1230.9200	1235.13	1230.4301	1233.0000	111929.0
2019-07-26 09:33:00	1233.0000	1237.90	1232.7500	1237.9000	86564.0
2019-07-26 09:34:00	1237.4449	1241.90	1237.0000	1241.9000	105884.0
2019-07-26 09:35:00	1241.9399	1244.49	1241.3500	1243.1300	74444.0

In [19]:

intraday_data['4. close'].plot(figsize=(10, 7))
# Define the label for the title of the figure
plt.title("Close Price", fontsize=16)
# Define the labels for x-axis and y-axis
plt.ylabel('Price', fontsize=14)
plt.xlabel('Time', fontsize=14)
# Plot the grid lines
plt.grid(which="major", color='k', linestyle='-.', linewidth=0.5)
plt.show()

Get data at a custom frequency

During strategy modelling, you are required to work with a custom frequency of stock market data such as 7 minutes or 35 minutes. This custom frequency candles are not provided by data vendors or web sources. In this case, you can use the pandas resample method to convert the stock market data to the frequency of your choice. The implementation of these is shown below where a 1-minute frequency data is converted to 10-minute frequency data.

The first step is to define the dictionary with the conversion logic. For example, to get the open value the first value will be used, to get the high value the maximum value will be used and so on.

In [ ]:

ohlcv_dict = {
 '1. open': 'first',
 '2. high': 'max',
 '3. low': 'min',
 '4. close': 'last',
 '5. volume': 'sum'
}

Convert the index to datetime timestamp as by default string is returned. Then call the resample method with the frequency such as

• 10T for 10 minutes,

• D for 1 day and

• M for 1 month

In [130]:

intraday_data.index = pd.to_datetime(intraday_data.index)
intraday_data_10 = intraday_data.resample('10T').agg(ohlcv_dict)
intraday_data_10.head()

Out[130]:

Date	Open	High	Low	Close	Volume
2019-07-17 09:30:00	1150.9200	1155.5500	1150.510	1154.28	82911.0
2019-07-17 09:40:00	1154.2950	1157.8400	1154.295	1157.76	35549.0
2019-07-17 09:50:00	1157.4250	1158.4399	1155.670	1155.67	30371.0
2019-07-17 10:00:00	1155.4700	1156.0200	1153.090	1153.09	21445.0
2019-07-17 10:10:00	1153.0194	1153.4200	1151.000	1152.29	31073.0

yfinance

yfinance is another module which can be used to fetch the minute level stock market data. It returns the stock market data for the last 7 days.

In [ ]:

# Install the yfinance if not already installed
!pip install yfinance

The yfinance module has the download method which can be used to download the stock market data. It takes the following parameters:

1. ticker: The name of the tickers you want the data for. If you want data for multiple tickers then separate them by space

2. period: The number of days/month of data required. The valid frequencies are 1d, 5d, 1mo, 3mo, 6mo, 1y, 2y, 5y, 10y, ytd, max

3. interval: The frequency of data. The valid intervals are 1m, 2m, 5m, 15m, 30m, 60m, 90m, 1h, 1d, 5d, 1wk, 1mo, 3mo

The below code fetches the stock market data for MSFT for the past 5 days of 1-minute frequency.

In [133]:

data = yf.download(tickers="MSFT",
 period="5d",
 interval="1m")
data.head()

[*********************100%***********************] 1 of 1 downloaded

Out[133]:

Datetime	Open	High	Low	Close	Adj Close	Volume
2019-07-17 09:30:00-04:00	137.70	137.75	137.23	137.33	137.33	645676
2019-07-17 09:31:00-04:00	137.33	137.43	137.22	137.40	137.40	112675
2019-07-17 09:32:00-04:00	137.39	137.40	137.18	137.29	137.29	73906
2019-07-17 09:33:00-04:00	137.44	137.58	137.39	137.42	137.42	127492
2019-07-17 09:34:00-04:00	137.44	137.52	137.43	137.45	137.45	56630

Stocks Fundamental Data

We have used yfinance to get the fundamental data. The first step is to set the ticker and then call the appropriate properties to get the right stock market data.

In [ ]:

# Install the yfinance if not already installed
!pip install yfinance

In [ ]:

# Import yfinance
import yfinance as yf
# Set the ticker as MSFT
msft = yf.Ticker("MSFT")

Key Ratios

You can fetch the latest price to book ratio and price to earnings ratio as shown below.

In [ ]:

# get price to book
pb = msft.info['priceToBook']
pe = msft.info['regularMarketPrice']/msft.info['epsTrailingTwelveMonths']
print('Price to Book Ratio is: %.2f' % pb)
print('Price to Earnings Ratio is: %.2f' % pe)

Revenues

In [134]:

# show revenues
revenue = msft.financials.loc['Total Revenue']
plt.bar(revenue.index, revenue.values)
plt.ylabel("Total Revenues")
plt.show()

Earnings Before Interest and Taxes

In [135]:

EBIT = msft.financials.loc['Earnings Before Interest and Taxes']
plt.bar(EBIT.index, EBIT.values)
plt.ylabel("EBIT")
plt.show()

Balance sheet, cash flows and other information

In [ ]:

# show income statement
msft.financials
# show balance heet
msft.balance_sheet
# show cashflow
msft.cashflow
# show other info
msft.info

Futures and Options (F&O) Data for Indian Equities

NSEpy

The nsepy package is used to get the stock market data for the futures and options for Indian stocks and indices.

Futures Data

In [15]:

from datetime import date
from nsepy import get_history
# Stock options (for index options, set index = True)
stock_fut = get_history(symbol="HDFC",
 start=date(2019, 1, 15),
 end=date(2019, 2, 1),
 futures=True,
 expiry_date=date(2019, 2, 28))
stock_fut.head()

Out[15]:

Date	Symbol	Expiry	Open	High	Low	Close	Last	Settle Price	Number of Contracts	Turnover	Open Interest	Change in OI	Underlying
2019-01-15	HDFC	2019-02-28	1986.70	2011.00	1982.95	2008.25	2006.20	2008.25	4810	4.796817e+09	2537500	2299500	1992.15
2019-01-16	HDFC	2019-02-28	2002.10	2010.15	1985.20	1992.15	1991.30	1992.15	2656	2.655748e+09	3783500	1246000	1975.00
2019-01-17	HDFC	2019-02-28	2003.60	2019.05	1991.60	2017.15	2013.00	2017.15	3993	4.008667e+09	5545000	1761500	NaN
2019-01-18	HDFC	2019-02-28	2018.55	2025.75	2005.00	2018.40	2017.25	2018.40	481	4.845300e+08	5637000	92000	2006.85
2019-01-21	HDFC	2019-02-28	2011.25	2031.10	1998.00	2016.55	2016.60	2016.55	1489	1.505249e+09	6258000	621000	2004.45

In [20]:

import matplotlib.pyplot as plt
stock_fut.Close.plot(figsize=(10, 5))
# Define the label for the title of the figure
plt.title("Close Price", fontsize=16)
# Define the labels for x-axis and y-axis
plt.ylabel('Price', fontsize=14)
plt.xlabel('Date', fontsize=14)
# Plot the grid lines
plt.grid(which="major", color='k', linestyle='-.', linewidth=0.5)
plt.show()

Options Data

In [17]:

from datetime import date
from nsepy import get_history
stock_opt = get_history(symbol="HDFC",
 start=date(2019, 1, 15),
 end=date(2019, 2, 1),
 option_type="CE",
 strike_price=2000,
 expiry_date=date(2019, 2, 28))
stock_opt.head()

Out[17]:

Date	Symbol	Expiry	Option Type	Strike Price	Open	High	Low	Close	Last	Settle Price	Number of Contracts	Turnover	Premium Turnover	Open Interest	Change in OI	Underlying
2019-01-15	HDFC	2019-02-28	CE	2000.0	52.70	56.00	52.70	56.0	56.0	56.0	3	3081000.0	81000.0	10000	1000	1992.15
2019-01-16	HDFC	2019-02-28	CE	2000.0	55.00	55.00	49.00	49.0	49.0	49.0	14	14358000.0	358000.0	11000	1000	1975.00
2019-01-17	HDFC	2019-02-28	CE	2000.0	59.15	64.65	51.00	61.9	61.9	61.9	27	27750000.0	750000.0	18500	7500	NaN
2019-01-18	HDFC	2019-02-28	CE	2000.0	63.00	63.00	60.00	60.0	60.0	60.0	7	7212000.0	212000.0	18500	0	2006.85
2019-01-21	HDFC	2019-02-28	CE	2000.0	62.05	69.00	62.05	62.9	62.9	62.9	6	6198000.0	198000.0	20000	1500	2004.45

In [21]:

import matplotlib.pyplot as plt
stock_opt.Close.plot(figsize=(10, 5))
# Define the label for the title of the figure
plt.title("Close Price", fontsize=16)
# Define the labels for x-axis and y-axis
plt.ylabel('Price', fontsize=14)
plt.xlabel('Date', fontsize=14)
# Plot the grid lines
plt.grid(which="major", color='k', linestyle='-.', linewidth=0.5)
plt.show()

Visualization and Analysis

After you have the stock market data, the next step is to create trading strategies and analyze the performance. I have created a simple buy and hold strategy for illustration purpose with four stocks namely Apple, Amazon, Microsoft and Walmart. To analyze the performance, you can use the pyfolio tear sheet as shown below.

In [ ]:

# Install pyfolio if not already installed
!pip install pyfolio

In [ ]:

import pyfolio as pf

In [16]:

# Define the ticker list
tickers_list = ['AAPL', 'AMZN', 'MSFT', 'WMT']
# Import pandas and create a placeholder for the data
import pandas as pd
data = pd.DataFrame(columns=tickers_list)
# Feth the data
import yfinance as yf
for ticker in tickers_list:
 data[ticker] = yf.download(ticker, period='5y',)['Adj Close']
# Compute the returns of individula stocks and then compute the daily mean returns.
# The mean return is the daily portfolio returns with the above four stocks.
data = data.pct_change().dropna().mean(axis=1)
# Print first 5 rows of the data
data.head()

[*********************100%***********************] 1 of 1 downloaded
[*********************100%***********************] 1 of 1 downloaded
[*********************100%***********************] 1 of 1 downloaded
[*********************100%***********************] 1 of 1 downloaded

Out[16]:

Date
2014-07-29 -0.003227
2014-07-30 -0.002592
2014-07-31 -0.020351
2014-08-01 -0.005142
2014-08-04 0.006887
dtype: float64

In [21]:

pf.create_full_tear_sheet(data)

Start date	2014-07-29
End date	2019-07-25
Total months	59
	Backtest
Annual return	26.6%
Cumulative returns	224.4%
Annual volatility	18.2%
Sharpe ratio	1.39
Calmar ratio	1.11
Stability	0.96
Max drawdown	-24.1%
Omega ratio	1.28
Sortino ratio	2.07
Skew	0.15
Kurtosis	4.25
Tail ratio	0.94
Daily value at risk	-2.2%

Worst drawdown periods	Net drawdown in %	Peak date	Valley date	Recovery date	Duration
0	24.08	2018-10-01	2018-12-24	2019-04-17	143
1	13.15	2015-07-30	2015-08-25	2015-10-23	62
2	13.12	2015-12-07	2016-02-09	2016-04-13	93
3	9.65	2019-05-03	2019-06-03	2019-06-18	33
4	8.46	2018-03-12	2018-04-02	2018-05-10	44

Stress Events	mean	min	max
Oct14	0.04%	-1.68%	2.17%
Fall2015	-0.17%	-4.67%	5.35%
New Normal	0.10%	-4.67%	7.17%

I hope you can use the Python codes to fetch the stock market data of your favourites stocks, build the strategies and analyze it. I would appreciate if you could share your thoughts and your comments below. Python is quite essential to understand data structures, data analysis, dealing with financial data, and for generating trading signals. For traders and quants who want to learn and use Python in trading, this bundle of courses is just perfect. Disclaimer: All investments and trading in the stock market involve risk. Any decisions to place trades in the financial markets, including trading in stock or options or other financial instruments is a personal decision that should only be made after thorough research, including a personal risk and financial assessment and the engagement of professional assistance to the extent you believe necessary. The trading strategies or related information mentioned in this article is for informational purposes only.

Reference : https://blog.quantinsti.com/stock-market-data-analysis-python/