Sometimes it may be a little bit frustrating when old codes do not work as intended, or do not work at all. A couple of years I have written a code for trading alerts. I have written many codes that has to with trading, stocks, crypto, indicators, etc. But this one in particular I wanted to reuse again. What the script does is goes through list of stock tickers, applies various indicators, and any of the indicators or combination of them trigger some sort of a signal (buy, sell, maybe buy, maybe sell, etc) it would create a list based on various categories. Initially this would be a longer list of stocks to consider, than these stocks would also go through a different script to come up with a shorter list. This was an experimental project, and I haven't used it in a while. I decided to revisit the code to see if it still works as expected, so I could reutilize parts of the functionality for something else. Unfortunately, the script didn't work at all, at first.
It is not a complicated code. Two main python libraries it uses are yfinance and finviz. Yfinance is one of the commonly used libraries to get historical data for stock tickers for free. I have used this library many times and it has been very reliable. There are many other options as well, but many platforms provide this data for fees. The second main library used in the code is Finviz. As the name may suggest, Finviz helps getting ticker symbols from Finviz website. Finviz website is a great resource for traders and investors. It has subscription plans as well that allows users use advanced features. For ordinary traders the information available for free on Finviz might be more than enough. One of the great things about Finviz is it lists almost all of the stock ticker symbols, in addition to all the financial information about the companies as well.
The first time I attempted to collect all the ticker symbols on Finviz was writing a scraping script and automating the website. At that time it did work without any issues. After all symbols are available I would use these ticker symbols to get the historical price data from yfinance. One of the issues was ticker symbols becoming obsolete due to being delisted from exchanges. Alternatives, newly added ticker symbols wouldn't be in the old list. For these reasons, the compiled list would need to be renewed once a week or so. Then I came across a library dedicated to Finviz, and was named finviz. I had to try it. When I did, I was impressed. Not only it did the job well, it had/has feature to filter the ticker symbols based all the options Finviz provides filtering by. It was super easy to use. It did/does many other things as well, but it worked great for what I needed it for.
So open the script and run it. I get errors right away. While I knew chances were the code wouldn't work right away, I still was disappointed. After going through errors, maybe I thoughts updating the libraries would do the trick. It didn't. I wasn't ready to rewrite the code or read through it all at that time. I wasn't going to give it one more try. It turned out something was from with the update for finviz that was breaking my old code. After finding sort of a solutions and using a patch, it seemed finviz was working ok, but now yfinance was giving errors. I wasn't ready to dedicate hours and hours going through the code. If simple update doesn't work, then it would probably better to leave it alone and rewrite the entire thing at a later time.
When I did find sometime to work on it and was prepared to rewrite everything, I found out there was nothing wrong with yfinance. This library continued to be reliable and was working just fine in tests. So the culprit was the finviz. Maybe it was time to let go of finviz and try something else. But I had to find out what may have been the issue out of curiosity. As I was searching for finviz documentation, I came across different Finviz libraries for python. One was pyfinviz, and another was finvizfinance. I got distracted and didn't even get a chance to go to the original library. I had to try the new ones.
After trying pyfinviz I was able to get the functionality I was looking for, which was getting all the ticker symbols. Because of this I didn't even explore finvizfinance yet. After getting this part fixed, I replaced the old finviz code in the script. Everything worked perfect! Success! Now that I got it to work, I can reutilize parts of the code elsewhere. This also means going forward I will be replacing old finviz library to this new pyfinviz. Pyfinviz doesn't have awesome feature finviz offered, but at least it works. In fact, Pyfinviz, doesn't do a lot. It has very limited functionality. We probably could write a scraping script and do everything Pyfinviz does and more with Beautiful Soup and/or Selenium. However, it is great to have tools and library available to get things done quickly.
Feel free to read the pyfinviz documentation yourself. It is a quick read. It can fetch news, crypto prices, insider trade information, checking ticker symbols, groups, and screener. What I am most interested in is the Screener. Remember it is not doing anything complicated. It is just grabbing all the information from Finviz website. Let's look at a sample code.
from pyfinviz.screener import Screener
# with no params (default screener table)
screener = Screener()
# with params (The first 3 pages of "STOCKS ONLY" where Analyst recommend a strong buy)
options = [Screener.IndustryOption.STOCKS_ONLY_EX_FUNDS, Screener.AnalystRecomOption.STRONG_BUY_1]
screener = Screener(filter_options=options, view_option=Screener.ViewOption.VALUATION,
pages=[x for x in range(1, 4)])
# available variables:
print(screener.main_url) # scraped URL
print(screener.soups) # beautiful soup object per page {1: soup, 2: soup, ...}
print(screener.data_frames) # table information in a pd.DataFrame object per page {1: table_df, 2, table_df, ...}
To get things moving, all I was interested in was all ticker symbols listed on Finviz. We can access them manually on their website. The problem is they only list 20 ticker symbols per page, and there are about 470 pages of them. To test if we can get all the tickers we can write a simple code like this.
stocks = []
screner = Screener(pages=[x for x in range(1,477)])
for i in range(1, 477):
ticker_list = list(screner.data_frames[i]['Ticker'])
stocks += ticker_list
print(stocks)
print(len(stocks))
The second line of code is where we are getting all the information. The information is return as a dictionary containing all pages, and each page is dataframe. There were 476 pages, that's why I am ending the range at 477 to get all pages including the last page. After that simply iterating through the result and adding tickers in my stocks variable. The end result is a list of 9511 ticker symbols.
The Crypto class of the Pyfinviz doesn't do a lot. It just returns a page with few crypto prices. It includes BTC, ETH, LTC, BCH, XRP, and I think that's it. That is because Finviz doesn't have extensive crypto information, I think. Which is ok, for crypto tickers there are other libraries we can utilize. Another useful feature Pyfinviz has is checking if a ticker symbol exists or not. Sometimes ticker symbols become obsolete due being delisted or mergers. Sometimes new ticker symbols are added. We can use Quote class to check the tickers.
It sounds like a massive headache and you were able to solve it by using a different library. It's nice to see all of these available and I hope your code won't have anymore issues.
Codes will always have issues. It is great to reolve them when problems appear.
Oh wow just wow 😲
Yeah it's so annoying when you need to update and change all stuff with new libraries, but unless they changed API or something it should still have worked the old script
Writing codes isn't a small task. You are doing a great job, thank you
So finally after a lot of hustle, the code started to work. and told it who is the boss here.
How do you get to write codes?
I know people who do so too but I’m always wondering how they will get to know what to write
You can read books, watch tutorials, or take some online classes to learn how to code.
It is true that trading is also not easy. There are many special things to be taken care of. If something goes wrong, then we suffer huge losses. Good thing is that now you understand all these things. Now you will get better profit from it.
Thanks, that is great!
I must confess, this is excellent brain work. You are a gem😃
I wish I was tech savvy enough to wrap my head round all these . Notwithstanding, thanks for sharing your code snippet and insights!
It's impressive how you efficiently retrieved and organized such a large amount of ticker symbols.
Pyfinviz sounds like a handy tool, especially for checking the validity of ticker symbols. Great job I must say!
Great job. Thanks for succeeding
Credit: antisocialist
Earn Crypto for your Memes @ HiveMe.me!
Credit: fun.miner
Earn Crypto for your Memes @ HiveMe.me!