Updates: lots of them in Site Updates (Josh)

Hi everyone!

I wanted to bring you all in the loop on everything that’s been going on – it’s a lot!

Mochi and I have been traveling for my dayjob for most of the past month, and he’s been the perfect little travel bear:

Now that things have calmed down a bit, I’m hoping to post shorter and more frequent updates to better communicate progress with the site.

Let’s start with some background
Before I dig in, I think it would be helpful to talk a bit about what’s been going on with the site over the last six months or so.

I don’t want to get too technical, so we’ll use this graph as a visual aid:

This is a graph of the memory usage of the site over the last 30 days. The higher bits are when more memory is being used, which usually equals the site running more slowly. Every time you see the graph dip a bit, that’s me manually restarting the site to avoid a crash. The crashes start happening without fail after about 3-4 days, and the site stays super slow at all times. I’ve written a script that has the server automatically restart every day at 2am PST. This is not a great fix, but it’s working as a duct tape-y band-aid for now.

Why is this happening?
That’s a great question! Through our detective work and SM’s detailed background info, we’ve all agreed that the likely culprit is the age of the underlying engine that powers PB. Think of it like the difference between the engine in a car built in 2000 and a car built today. There have been tremendous strides in efficiency over the years, and while the old car might still function, the newer model is much more efficient.

Okay, you think you’ve figured it out. Why haven’t you fixed it yet?
As I touched on in a previous post, PB is a completely custom creation. The way sites like PB are built today relies a lot on what are called software packages. Those are libraries of code that do a particular thing very well. The advantage of using these is that you don’t have to reinvent the wheel when building, say, login functionality. The flip side is that when these become out of date, you never quite know what’s going to break elsewhere on the site when you update them. A lot could go wrong if we just flip the switch and bring everything up to date.

Slow and steady is going to win the race here, but it’s going to be vastly easier to keep up with any updates once we have a solid foundation to work from. I’m confident that at the end of these fixes we’ll have with a fast, stable PB that will be solid for years to come.

So, when are you getting started?
Well, that’s the rub. The more that Lt Dan dug into the architecture, the more he realized that his skillset isn’t the best fit for this very necessary work. He feels that while he could take the time to figure out the skills needed to upgrade and fix the server, the site would be better served if I use any resources I’m able to marshal to contract with an expert. I agree, and right now I’m exploring alternatives for the server work.

So, a bit of a setback.

I’ve reached out to a former colleague with expertise in this area, and he seems interested. Nothing is formalized yet, but it’s looking positive. That said, if you or someone you know has skills in this area (DevOps), let’s chat. I’m always available at [email protected]

Experts cost money, how are you going to pay for this?
I’m still feeling out what that would look like, but once the scale of the work is nailed down and I get an estimate, I think a very focused and transparent site-wide fundraiser could make sense. Anything that might come in over the total amount that we need would be stored for future emergencies. Then, after we have the initial updates taken care of, we’ll start to have the conversation about paid subscriptions and extra features.

How does that feel to all of you? If there’s anything you think I’m missing, I’m very open to suggestions.

More soon!

<3,
-J

Last updated May 09, 2018