Welcome to the new website!

Introduction

HODP's goals are fairly simple.

1. Build cool stuff with open data.
2. Get people to see it.
3. Inspire them to take action.

Raising awareness is key for both of these processes.

Each of these goals fuels the others. The more cool projects we publish, the more traction we get, which in turn helps us promote data-driven policies to the university.

In the past, we've kept these components separate, publishing articles on Medium and hosting web apps and data on our website. While Medium made publishing convenient, it made it difficult for people who read our articles to explore our other work.

This past June, we decided it was time to bring the pieces together. We spent the summer redesigning and re-implementing the website from the ground up, building native articles with the flexibility of web apps.

Here's what's new:

• Customizable articles: We can now code interactive articles and visualizations. Data analysis adds rigor to storytelling, and writing helps us make intuitive sense of the data. We can create more engaging stories by bringing the best of web-apps and articles together.
• Revamped design system: We meticulously chose fonts, colors, and spacing to convey a clear information hierarchy within the site.
• All of HODP in one place: It’s difficult to grow readership when we host text articles on Medium and nest web apps under the data catalog. Now, you can read interactive stories, play the predictions game, and search for datasets all from one platform.
• The upshot: If we create cooler projects and make them easier to find, more people will discover them. If more people understand what we do, more students will sign up for the survey group, which will bolster future projects. But more importantly, we hope that readers will get excited about working with open data themselves.

Decisions, decisions

With the vision in mind, we needed to figure out the implementation. Our first decision was to start from scratch instead of editing our previous site, which was written in Flask and templated in Jinja. Switching the frontend to React was a natural choice: each of our core components has several configurations, and we could encode those as props. React was also the most commonly known JS framework among board members, and we wanted to minimize the barriers to working on the site.

After deciding on React, we needed to figure out how to store our articles. Markdown was out, because we needed custom code. We ultimately decided on a headless CMS, which is a system that stores content without dictating its presentation. (Read: they give you an API; you write the frontend yourself.) After lurking on StackShare and forking some starter repos, we decided on Sanity.io. I wrote a schema to store information about articles, blog posts, and other page content, and then I built an embedded React component type into our article model. This infrastructure allows HODP members to code interactive stories and manage them from the Sanity Studio.

Our codebase stores very few images and text blocks on its own: instead, we pull the content from Sanity. As we discovered with our previous website, it’s a pain to submit a PR just to add a contributor or replace a featured article. Sanity provides an interface to edit our content while leaving us the freedom to control the presentation.

We chose Gatsby as a site generator to interface between Sanity and our React code. With Gatsby, we could set up our Sanity data layer and write content queries in GraphQL. More importantly, Gatsby pre-generates raw HTML / CSS / JS from our React code, which optimizes our site for speed and SEO.  Hopefully you can confirm that this site is truly “blazing fast.”

More than a coding project

During our website planning meeting at the beginning of the summer, we mapped out user flows alongside HODP goals. (For the record, our users are “interested Harvard students,” “HODP members,” “potential partners,” and “randos.”). Some wild ideas made it onto the brainstorming doc — building an IDE inside HODP, anyone? I left the Zoom meeting with a sense of buoyant idealism.

An early planning doc.

Then, I built a basic version of the website and realized that it looked bad. Not terrible, but nowhere near our aspirational Google doc. The body copy looked sloppy, the margins were huge (ostensibly to make up for the lack of substance), and the article layout somehow looked mediocre at all sizes.

Ultimately, we realized that we couldn’t design and code simultaneously and expect a good result. I ended up sitting down with Figma for a weekend and redesigning the website from the ground up, from the style guide to the component hierarchy.

The new style guide.

Leo and I had long conversations about fonts.

Site mock-ups for pages, graphics, and article layouts. We needed to design a layout that used horizontal space effectively but could stack elegantly on mobile.

Anyways, I got really excited about this Figma design. Then my internship got busy, and I procrastinated for a month.

Activating serious mode

Fast forward to mid-July. I implemented the design, schemas, and project page generation. Kevin built in article comments, the MailChimp embed, and an interactive dashboard to visualize COVID-19. Alex built search and filtering into the data catalog, Sahana built the all things People, Matthew planned the Firebase infrastructure, and Daniela designed the homepage.

After completing those features, we met as a board to migrate all our past content from Medium to Sanity. We had only ~50 articles to transfer and 16 board members to help, but the process was still more tedious than expected. (We also needed to add roughly 70 HODP contributors, whose headshots we sourced from Facebook and LinkedIn.)

After months of work, roughly 150k API requests, and 200 Netlify build minutes... we made it!

We now have native articles, live-updating maps, and plug-in content that doesn't require us to submit a PR to fix a typo. We've come a long way since our original website.

I hope this was helpful for future HODP members and / or other student organizations that want to undertake similar projects. If you’re looking for the code, this project is open source.

Takeaways

When you don't start with a playbook, you get to design your own rules. Instead of working within an existing framework, you can reexamine the premises. Instead of figuring out how, you get to take a step back and ask why. In our case: instead of screenshotting a graph to meet Medium constraints, why not build NYT-style scrollytelling? By starting from scratch, we could set the parameters that future users will take for granted. Our code is far from complete: it’s a part-present solution, part-hypothesis about what we’ll need in the times ahead.

Here are some of our takeaways from that process.

• Design for flexibility. When planning each schema, we asked ourselves how the data might change in five years. What if the board structure changes, or what if we want to build dataset preview pages? We designed the content model to adapt to plug-in edits without causing major breaking changes.
• Spend less time evaluating, more time doing. We agonized over some of those early decisions — Sanity vs. Strapi, Post Grotesk vs. Atlas Grotesk, etc. It turns out that documentation and other people’s opinions can only take you so far. Ultimately, we saved time by taking first steps in several directions rather than overthinking where to make the first move.
• Remember the 80/20 rule. It took one day to set up custom components in our content model, but it took weeks to build the features we take for granted: mobile responsiveness, footer elements, internal links, project pagination, and more. So many times we declared that we were “almost there,” yet we had so much left to do. I’ll add the planning fallacy to the list of principles to keep in mind, whose guiding principle is “remember that you will forget the 80/20 rule.”
• Trust that you’ll figure it out along the way. Before writing any code, I thought that I needed a plan that could handle every edge case. Truth is, if I had known about all the edge cases we would discover in the process, I wouldn’t have known where to start. Don’t let uncertainty paralyze you into inaction: your future self will be in a much better epistemic position to answer those questions than your current self. Ride the wave of enthusiasm and know that you’ll develop the expertise along the way.

Zooming out

In the year since Kevin (Bi), Lucy, Seth, and I first discussed moving articles off Medium, HODP has started a survey group, a predictions game, and a wiki. These initiatives were all started on a whim, but they quickly became some of our most impactful projects.

We don’t have a set agenda each year, except to teach a bootcamp and see what happens. HODP gives you the freedom to ask “what if…” and the skills to follow through with it. There aren’t many clubs that offer that much agency — I think that’s what makes HODP special.

To HODP members: we’ve created a platform that allows you to publish cool stuff. People will see it, and it will make an impact. The last part is up to you.

I'd like to thank:

• Jason Kao from Columbia, for inspiring us with his work on the Spec.
• Joshua Pan for early feedback and advice.
• FiveThirtyEight, the New York Times, and The Atlantic for design inspiration.
• The Sanity Slack channel (particularly @knut).
• Leo for his stunning cover photos and for changing his mind about Post Grotesk.
• Sahana for beautifully documented PRs and for all-around competence.
• Ashley for offering a sounding board (and for telling me to sleep when I overlapped too much with Sydney time.)
• The Tech Squad (Alex, Matthew, Kevin, and Daniela) for being incredible friends and teammates. I’m excited to see where we take this next!

If you're interested in working on the website, please reach out to me on Slack :)