The John Muir Trail, including a Half Dome side trip, ranks as the most difficult hike in the United States to get a permit for. As you may remember from our selfie-a-day compilation, @trang and I hiked the trail prime-season, starting on August 28, 2018. Today, I'm open sourcing the bot I wrote and put to the test to secure our JMT permits earlier this year.
The bot is named hackjohn and is for finding southbound JMT permit spaces. Hiking southbound — starting in Yosemite National Park and finishing at Mount Whitney — is the preferred direction to hike the JMT due to the easier logistics and more gradual acclimation to high altitude.
Southbound JMT Permits
At the core, what you need to legally hike the JMT Southbound is a Yosemite wilderness permit. Ours looked like this:
The permit is a piece of paper you pick up from a wilderness permit station. But here's the difficultly, Yosemite only issues 45 permits a day for exiting the park via Donahue Pass into the Ansel Adams Wilderness. 10 of these 45 spaces are reserved for walk-up permits leaving from the Lyell Canyon Trailhead in Tuolumne Meadows, which skips the first 20 or so miles of the trail. Therefore, if all else fails, keep in mind that a walk-up permit is likely possible… high risk, high reward. Walk-up permits are also possible for the other Yosemite trailheads, but only if there is a cancellation… e.g. someone doesn't show up by 10 AM to pickup their permit (and I think these can also be reserved by phone).
But if you'd like one of the 35 guaranteed permit reservations per day, you have two options:
Enter the lottery drawing to initially fill the 35 spaces, which occurs 24 weeks (168 days) in advance of the start date. It looks like Yosemite now has an online system for entering the lottery, which will be more convenient (and thus even more competitive) than the fax-based submission submission of the past. The lottery application allows you to specify a 21 day range of start dates. If you want to win the lottery, start date flexibility is a must. Prepare yourself for the dreaded daily emailed titled "Denied: Yosemite Wilderness Lottery", which reads:
We regret to inform you that your application was not selected in today’s wilderness permit lottery. This application will roll to tomorrow’s daily lottery, and to each day after, until your application request is granted or date range expires. Once the date range expires, you may reapply with a new application and date range. Please only send one application per group and date range.
The main problems with the lottery approach are the long lead time (24 weeks) and low acceptance rate requiring entrants to settle on sub-optimal starting trailheads and dates. As such, it's no surprise that many individuals / parties win a lottery spot that they end up not being able to hike. This leads to option two.
Unless you want to hike the trail super early or late in the season, at which point you had better be prepared for Hell in the High Sierras, all 35 spaces will be initially assigned via the lottery. Yosemite provides a "Full Trailhead Report" webpage with a table titled "Donohue Exit Quota and Trailhead Space Available". When a space becomes available, either because the lottery did not fill it or because of a cancellation or party size change, this table is updated. If you see a vacancy in this table that you'd like, call Yosemite Wilderness Permit Reservations at 1-209-372-0740 immediately. If someone else hasn't already snatched it, you'll be able to make the reservation on the spot over the phone. This is substantially easier than competing for the chance to get a reservation via the lottery.
Introducing Hackjohn
As you may have guessed, hackjohn is a bot to monitor for available spaces in the Donohue Exit Quota and Trailhead table online. Essentially, it allows you to take advantage of method 2 above for getting a JMT permit without having to manually check the webpage every day at exactly the right time — the table is updated around 11 AM Pacific Time. Hackjohn allows you to specify the starting dates, trailheads, and group size that you're looking for. If it detects space, it will send you a notification via Telegram Messenger.
At this point, users must run hackjohn for themselves. The source code is available at https://github.com/dhimmel/hackjohn under the permissive MIT License. Configuring hackjohn requires the following steps:
- Cloning the GitHub repository, i.e.
git clone https://github.com/dhimmel/hackjohn.git - Installing the conda environment with the required dependencies (as per the README)
- Editing the configuration variables in
hackjohn.py, specificallyspaces(for number of spots required),exclude(for trailheads to exclude), anddates(the range of desired dates). - Generating a token for receiving the Telegram notifications by messaging MiddleManBot
- Scheduling
python hackjohn.pyto run around 11 AM Pacific Time. The README describes scheduling via thecronutility and writing the output to a local log file.
If all goes well, you will be awaiting a Telegram message like this:
If you see this message call immediately, as I've heard that many vacancies last only a matter of minutes.
In my case, I ended up using hackjohn to change my reservation several times. I had initially gotten a permit via lottery for September 13, which was too late as I was hoping to bundle the hike with a trip to California for a friend's wedding. Therefore, I developed hackjohn and ended up moving my start date sooner whenever a more preferable spot became available. However, @trang decided to join for the hike a mere 7 weeks before the start. Therefore, we had to further use hackjohn to get us permits for the same trailhead on the same day. We ended up getting to Yosemite on August 28 around noon with permits starting on the 29th. Due to the forest fires (which didn't end up affecting our hike at all), there were plenty of vacancies and we could start that day rather than paying exact cash to stay in the backpacker's campground in the Valley for the first night.
As our view of Sallie Keyes Lake demonstrated, the time developing hackjohn paid off:
Utopian Information
I'm labeling this post as a development submission to @utopian-io, which is a Steem community to reward open source software contributions. This submission is for the new project at https://github.com/dhimmel/hackjohn and my username on GitHub is dhimmel (same as on Steem).
Regarding the technicalities of the Python implementation, the heavy lifting to extract the trailhead space table is done using Pandas, which I contribute to from time to time. From hackjohn.py:
url = 'https://www.nps.gov/yose/planyourvisit/fulltrailheads.htm'
response = requests.get(url)
response.raise_for_status()
wide_df, = pandas.read_html(
response.text,
header=1,
attrs = {'id': 'cs_idLayout2'},
flavor='html5lib',
parse_dates=['Date'],
)
wide_df = wide_df.iloc[:, :6]
trailhead_df = wide_df.melt(id_vars='Date', var_name='Trailhead', value_name='Spaces')
trailhead_df = trailhead_df.dropna()
trailhead_df.Spaces = trailhead_df.Spaces.astype(int)
Without Pandas, extracting and reformatting the HTML table would take anywhere from 10 to 100 times the lines of codes. And the more lines of code you must write, the greater the chance of a bug going unnoticed. Much respect to the two dataframe functions read_html and melt!
Most things went smoothly, except when hackjohn started producing bogus output for an unknown reason. Turns out I had updated my Pandas version to 0.23, which introduced a read_html bug that I reported here. This experience is why you should always create a versioned environment for any type of analysis script if you want it to continue working the future. Hackjohn uses conda to version the critical dependencies. However, I know not all users may take the time to setup an environment just for hackjohn, so I included the following check to make sure hackjohn fails with a clear error message when the toxic Pandas version is imported.
from pkg_resources import parse_version
pandas_version = parse_version(pandas.__version__)._version.release
if pandas_version[:2] == (0, 23):
# read_html malfunctions in pandas v0.23
# https://github.com/pandas-dev/pandas/issues/22135
raise ImportError('pandas v0.23 is not supported due to https://git.io/fp9Zn')
Anyways, please chime in with any technical or help needed questions or open a GitHub Issue. Happy hiking and see you on the trail.
License
This post as well as its images are released under a CC BY 4.0 License. Republish or reuse this content, but please attribute by linking to this Steem post by @dhimmel.
Great post to define the problem and the challenge, as well as the finding the solution.
The check about the pandas version is a great touch.
Can't agree. Crontab's interface is amazing and works very well in comparison to hipster solutions. :)
I would also like to see a installation instruction without conda. (Packacing it with setup.py or at least a requirements.txt) Conda is not a industry standard yet, I code with Python almost ten years, never used it.)
Also, reading the configurational variables outside of the logic may be good for other people. (Ex: environment variables.)
It's a minor thing but looks like you also have a redundant variable call here.
Your contribution has been evaluated according to Utopian policies and guidelines, as well as a predefined set of questions pertaining to the category.
To view those questions and the relevant answers related to your post, click here.
Need help? Write a ticket on https://support.utopian.io/.
Chat with us on Discord.
[utopian-moderator]
Thanks @emrebeyler for the thorough and helpful review. Much appreciated.
I added support for virtual environment with pip installation of
requirements.txtin this commit. I usually work on scientific computing projects where there are more heavyweight dependencies, often written in other languages then Python, socondamakes the most sense. But here the dependencies are all Python and virtual env does a fantastic job.I did consider making the script into more a package... e.g. adding
setup.pyand making it a command line utility withargparseargument parsing. However, I reckoned that many users will need to modify the source code to suit their custom needs and at this point the added complexity would be more of a barrier to customization.My gripe is that I have trouble remember what the following cron config means:
Also placing minutes before hours seems wrong.
Remnant from when I was writing this code in Jupyter notebook cells!
Thank you for your review, @emrebeyler! Keep up the good work!
This is classic. I would love to do it one day but not anytime soon.
Yes! It was a great time, although the world doesn't need more JMT pictures.
What's the longest hike you've done?
Hmm. Longest was in Nepal; about 25 days without a shower.
Hackjohn in the News
An exciting update: Lisa Krieger wrote a story on Hackjohn for The Mercury, a San Jose based newspaper. Published on April 22, 2019, the story is titled Here’s how to get a coveted reservation to hike the John Muir Trail. Lisa interviewed both @trang and I and links to both the project's GitHub and Steem post! Always good to see a journalist link thoroughly to sources.
Now this is cool! I know you are an avid hiker and IT-pro but I would never think you can combine these two, seemingly incompatible things :) Good job Daniel.
Quid pro quo
Should you find the hackjohn bot helpful in securing a permit for the John Muir Trail, I have a small request in return. PLEASE LEAVE NO TRACE. In fact, even better leave a negative trace and pick-up any litter you see along the way as @trang and I did on our thru-hike.
Also avoid practices that create litter, like packing too many items, being disorganized, or god forbid dangling all your belongings off your backpack. Swallow your dish water. Vaporize toothpaste ejections such that no globs hit the ground. Do not bury your wet-wipes as they do not decompose. In fact, on the JMT you're supposed to carry out toilet paper, so consider experimenting with the backcountry bidet.
Congrats on the utopian-io upvote!
Hi @dhimmel, you have received an upvote from trang. I'm the Vietnamese Community bot developed by witness @quochuy and powered by community SP delegations.
Hi, @dhimmel!
You just got a 100% upvote from SteemPlus!
To get higher upvotes, earn more SteemPlus Points (SPP). On your Steemit wallet, check your SPP balance and click on "How to earn SPP?" to find out all the ways to earn.
If you're not using SteemPlus yet, please check our last posts in here to see the many ways in which SteemPlus can improve your Steem experience on Steemit and Busy.
Congratulations @dhimmel! You have completed the following achievement on the Steem blockchain and have been rewarded with new badge(s) :
Click here to view your Board of Honor
If you no longer want to receive notifications, reply to this comment with the word
STOPDo not miss the last post from @steemitboard:
Hi @dhimmel!
Your post was upvoted by @steem-ua, new Steem dApp, using UserAuthority for algorithmic post curation!
Your post is eligible for our upvote, thanks to our collaboration with @utopian-io!
Feel free to join our @steem-ua Discord server
Hey, @dhimmel!
Thanks for contributing on Utopian.
We’re already looking forward to your next contribution!
Get higher incentives and support Utopian.io!
Simply set @utopian.pay as a 5% (or higher) payout beneficiary on your contribution post (via SteemPlus or Steeditor).
Want to chat? Join us on Discord https://discord.gg/h52nFrV.
Vote for Utopian Witness!