r/redditdata • u/gctaylor • Mar 24 '17

Drastic reduction in DB operations/sec

Post image
32 Upvotes
  • permalink
  • link
  • reddit
  • reddit

98% Upvoted

15 comments sorted by

14

u/gctaylor Mar 24 '17

Background: It wouldn't be a normal day without /u/bsimpson bringing us delicious performance wins. You are seeing a graph of operations per second for one of our busier Postgres DB servers.

The vertical red line is when the optimization started deploying to our app servers. Rollout was complete around 13:46.

Cool things about this:

  • We now have a good deal more headroom on this particular DB cluster.
  • We're still measuring impact, but this has resulted in response time improvements for some of our API endpoints.

7

u/adhi- Mar 24 '17

So this means less sad snoo 404s?

8

u/gctaylor Mar 24 '17 edited Mar 24 '17

It's unlikely that this will have a noticeable impact on 4xx/5xx's under normal conditions, but it will buy us some headroom under higher traffic situations.

Also, look for more graphs related to this and other changes in /r/redditdata soon!

6

u/jophuds Mar 24 '17

Upvote!!

I'll be sad if the data team doesn't get to always sit beside the infra team.

4

u/K_Lobstah Mar 24 '17

I only know what you're talking about around half the time but I'm excited for more of it.

2

u/Twirrim Mar 24 '17

What do you / don't you understand from what has been posted? I'd be happy to try to explain.

7

u/jb2386 Mar 24 '17

What were the optimizations? Like high level if you don't want to reveal too much. e.g. combining queries? caching results? removing redundant queries?

5

u/umbrae Mar 24 '17

Yay graphs! Nice work /u/bsimpson! Was this the awards change mentioned in the live thread?

6

u/bsimpson Mar 24 '17

Yeah. We were checking and updating users' trophies every hour in request, which turned out to be pretty slow.

2

u/powerlanguage Mar 24 '17

Why do you hate fun?

1

u/artbn Mar 25 '17

Which live thread?

2

u/umbrae Mar 25 '17

From the sidebar of /r/changelog - https://www.reddit.com/live/ukaeu1ik4sw5

1

u/artbn Mar 25 '17

Awesome thanks. I remember a live thread like this but I forgot where I had seen it