đź” Future update: Groups + Roadmap
Big wall of text incoming, I’ve been busy.
What’s next?
The next planned features for CougarGrades include the “Groups” page (temporary name). This page has been greyed out and marked with a “🔒” in the navigation bar since launch. The intended use of this feature is to compare courses in a specific category or “Group”. Specifically, I’m referring to the UH Core Curriculum by Component Area. These Component Areas are what most UH students are probably familiar with, such as: Life & Physical Sciences, Language, Philosophy, & Culture, Writing in the Disciplines, and many others.
Road Map
Here’s a rough road map of what I’d like to have prepared before introducing “Groups”. The ones that are checked are already done, the “-“ indicate W.I.P.
- @cougargrades/api v2.0.0
- @cougargrades/importer v2.0.0
- Port website to TypeScript
- Redesign website to have less bloat
- Build consistent pipeline for additional data (see: @cougargrades/publicdata)
- Use new data and systems for “Groups” feature
Progress so far
Technical mumbo jumbo ahead:
A big challenge of introducing new data into CougarGrades is that importing and indexing over 100,000 rows of CSV into NoSQL can be a challenge when it comes to timing. As I’ve noted before, past database upgrades have taken up to 12 hours or more to complete using the legacy Python importer tool. This is because the tool ran synchronously where each row of data was ran one at a time and required a multiple round-trips to verify that duplicates weren’t being created (very slow).
I’m proud to say that this is not the case anymore. As part of the API 2.0.0, future imports are done entirely in the cloud and are processed asynchronously by serverless, idempotent, cloud functions (SUPER fast).
Paired with a rewritten Importer tool in TypeScript, the entire CougarGrades database can be imported and calculated in less than 45 minutes instead of 12 hours or more.
Here’s a demo:
This was a lot of work and there were a lot of challenges along the way. For example, I needed to figure out how to incrementally calculate standard deviation without accessing any of the other elements at runtime. For my first dive into concurrent computing, I’m really proud of how it turned out.
The Present + Future
The current version of the website (0.4.3) will stay published throughout this first round of Fall 2020 course registration to ensure stability and uptime. I’m planning on deploying to production when the “Groups” feature is ready. I want to take my time and get it right. Thankfully, there’s a lot of time until August when the next round of course registration runs through.
Thanks for using CougarGrades! Stay safe and good luck on finals.
Enjoy your Summer! ⛱️🌴