• your-recovery-is-possible

    your-recovery-is-possible asked:

    Is Tumblr really shutting down? !

    Nope! Nope nope nope. Any time you see a reblog chain that claims something outlandish like that, it’s a good time to be skeptical — even if they sneakily tried to make it seem like the original post was from Tumblr Staff.

  • engineering:

Who doesn’t love animated GIFs?
Believe it or not, support for GIFs at Tumblr was a happy accident! When Tumblr put together the code for handling JPEGs, support and GIFs (and PNGs) happened to also work using the same code. Perhaps even more surprising is that the tools used to handle GIFs at Tumblr hadn’t changed much from those early days. 
The image above is an original from sukme that could not be posted to Tumblr last June. It also would have failed if he’d tried last Sunday. If you click-through to the original post, you will see a muddy, reduced-saturation mess. All this because our resizer couldn’t handle the original. 
I’ve got ninety-nine problems and the GIF is one
There is a lot of misinformation about GIF limits on Tumblr, so let me set the record straight: We don’t count colors or frames or pixels. We only count bytes and seconds. Every image that comes in is scaled to a number of smaller sizes and the smaller your image is, the fewer resizes need to happen, which means less time. 
We had two core failure modes in our prior resizer: Some images would take as much as several minutes to convert. This was not directly attributable to color, dimensions, or frame count, but a mysterious mix of all of them. Some images would balloon in size (600KB at 400x400, 27MB at 250x250).
The unpredictability of these failures made our GIF limits feel arbitrary and terrible to the end users. Some have gone so far as to threaten monkey kicks. I don’t want to get kicked by a monkey, so we started working hard late last year to fix it. 
A proposed solution
Some of you may have seen this post where the performance of our current converter was compared with a new “mystery” converter. The mystery converter was roughly 1000x faster on the “slapping” GIF and happened to look great, but had quality problems on other images. Those were more fully explored in here a couple of days later.
If you haven’t figured it out yet, the mystery converter is gifsicle.
Getting a better handle on it
To get an unbiased test set, I took a random sample of roughly 90K GIFs that Tumblr users tried to upload, not limiting the corpus only to those that succeeded. These were tested against the current converter, resizing down to the next size we produce. Each resize is given up to 20 seconds to complete in our application, but all resizes must complete in 30 seconds. All resizes must be under 1MB or we will convert the first frame to JPEG and call it a day. 
2.6% of my 90K GIFs took longer than 20 seconds to resize. This is an underestimation of how many GIFs would be rejected for time because this is only one of several resizes required. A whopping 17.1% of all GIFs were over 1MB. Even if we bump up to 2MB, the rejection rate is 2.75%. The converter was making over 25% of all resizes larger than the higher-resolution originals! The total rejection rate for my sample set was 4.46% of all original GIFs uploaded. 
Using gifsicle is so much faster that our CPU rejection rate drops to 0.00 on my test set. Also, just under 99% of all images were smaller when resized than they were at their original resolution. The size rejection rate was a much lower 0.59%.
Gifsicle problems
As compelling as the performance of gifsicle is, the quality problems are too much to ignore. We played around with the code a bit, but eventually we just got in touch with the author, Dr. Eddie Kohler. The specifics are in this post, but the short version is that Eddie was able to improve quality by adding some more advanced resampling methods as well as palette expansion for small-palette images. This increased our size rejection rate to 0.68% while still keeping us well under our CPU budget. 
Proving it
Image processing is all about choices. How do you resample? Do you sharpen? Where in the workflow is gamma correction applied, if at all? The list goes on and on. 
As you can imagine from the performance differences, our previous converter and gifsicle take very different approaches to GIF resizing. The output images look different. Sometimes it is slight, sometimes it is significant, but there is no way we could put out a converter that messes up your images, even if it messes them up quickly. 
We set up a qualitative study. The goal was simply to prove that we weren’t doing worse than our old converter, not necessarily that we were doing better. This study was opened up to all Tumblr employees, as well as some “randomly selected” outsiders (my friends and family). Participants were presented with one of two questions:
1.) Given an original and 1 resize, decide whether it is ok, unacceptable, or completely broken.
2.) Given an original and 2 resizes (randomly choses which was left and which was right, sometimes they were identical), choose the better image or say there is no difference.
The results were everything I could have hoped for. The “acceptable” test showed that users found gifsicle better at producing acceptable results (87% vs. 84%), but not by a statistically relevant amount (p=0.086) and that gifsicle produced fewer broken GIFs (0.71% vs. 1.38%), but again not enough to say it is definitively better (p=0.106). The “better” test found users preferring gifsicle 37% of the time, the prior converter only 16% of the time, but users also preferred one identical image over the other 27% of the time. Again, it is hard to say that gifsicle is better, but it is clear that it is no worse.
Putting it all together
The development and testing described above took from late October until the beginning of March. Packaging, deployment, and integration took only a couple of weeks!
We aren’t done. There is work underway exploring how we handle JPEGs and PNGs. There are a slew of features that we can go after. This was a big step, a necessary step, but not the end for sure. 
We are a community, it takes a village, there’s no “i” in GIF
This project couldn’t have happened without the excellent work of Eddie Kohler in creating, maintaining, and enhancing gifsicle. Tumblr’s Site Reliability Engineering group packaged and helped deploy gifsicle onto hundreds and hundreds of machines in our datacenter. Tumblr’s Security Team vetted the code, both by inspection and by attacking it to make sure we stay safe. This was all for the awesome Tumblr creators, but I have to mention qilme/sukme (same dude, two blogs), reallivingartist, and especially gnumblr for their help in understanding and ultimately attacking this monstrous problem.

    engineering:

    Who doesn’t love animated GIFs?

    Believe it or not, support for GIFs at Tumblr was a happy accident! When Tumblr put together the code for handling JPEGs, support and GIFs (and PNGs) happened to also work using the same code. Perhaps even more surprising is that the tools used to handle GIFs at Tumblr hadn’t changed much from those early days. 

    The image above is an original from sukme that could not be posted to Tumblr last June. It also would have failed if he’d tried last Sunday. If you click-through to the original post, you will see a muddy, reduced-saturation mess. All this because our resizer couldn’t handle the original. 

    I’ve got ninety-nine problems and the GIF is one

    There is a lot of misinformation about GIF limits on Tumblr, so let me set the record straight: We don’t count colors or frames or pixels. We only count bytes and seconds. Every image that comes in is scaled to a number of smaller sizes and the smaller your image is, the fewer resizes need to happen, which means less time. 

    We had two core failure modes in our prior resizer: Some images would take as much as several minutes to convert. This was not directly attributable to color, dimensions, or frame count, but a mysterious mix of all of them. Some images would balloon in size (600KB at 400x400, 27MB at 250x250).

    The unpredictability of these failures made our GIF limits feel arbitrary and terrible to the end users. Some have gone so far as to threaten monkey kicks. I don’t want to get kicked by a monkey, so we started working hard late last year to fix it. 

    A proposed solution

    Some of you may have seen this post where the performance of our current converter was compared with a new “mystery” converter. The mystery converter was roughly 1000x faster on the “slapping” GIF and happened to look great, but had quality problems on other images. Those were more fully explored in here a couple of days later.

    If you haven’t figured it out yet, the mystery converter is gifsicle.

    Getting a better handle on it

    To get an unbiased test set, I took a random sample of roughly 90K GIFs that Tumblr users tried to upload, not limiting the corpus only to those that succeeded. These were tested against the current converter, resizing down to the next size we produce. Each resize is given up to 20 seconds to complete in our application, but all resizes must complete in 30 seconds. All resizes must be under 1MB or we will convert the first frame to JPEG and call it a day. 

    2.6% of my 90K GIFs took longer than 20 seconds to resize. This is an underestimation of how many GIFs would be rejected for time because this is only one of several resizes required. A whopping 17.1% of all GIFs were over 1MB. Even if we bump up to 2MB, the rejection rate is 2.75%. The converter was making over 25% of all resizes larger than the higher-resolution originals! The total rejection rate for my sample set was 4.46% of all original GIFs uploaded. 

    Using gifsicle is so much faster that our CPU rejection rate drops to 0.00 on my test set. Also, just under 99% of all images were smaller when resized than they were at their original resolution. The size rejection rate was a much lower 0.59%.

    Gifsicle problems

    As compelling as the performance of gifsicle is, the quality problems are too much to ignore. We played around with the code a bit, but eventually we just got in touch with the author, Dr. Eddie Kohler. The specifics are in this post, but the short version is that Eddie was able to improve quality by adding some more advanced resampling methods as well as palette expansion for small-palette images. This increased our size rejection rate to 0.68% while still keeping us well under our CPU budget. 

    Proving it

    Image processing is all about choices. How do you resample? Do you sharpen? Where in the workflow is gamma correction applied, if at all? The list goes on and on. 

    As you can imagine from the performance differences, our previous converter and gifsicle take very different approaches to GIF resizing. The output images look different. Sometimes it is slight, sometimes it is significant, but there is no way we could put out a converter that messes up your images, even if it messes them up quickly. 

    We set up a qualitative study. The goal was simply to prove that we weren’t doing worse than our old converter, not necessarily that we were doing better. This study was opened up to all Tumblr employees, as well as some “randomly selected” outsiders (my friends and family). Participants were presented with one of two questions:

    1.) Given an original and 1 resize, decide whether it is ok, unacceptable, or completely broken.

    2.) Given an original and 2 resizes (randomly choses which was left and which was right, sometimes they were identical), choose the better image or say there is no difference.

    The results were everything I could have hoped for. The “acceptable” test showed that users found gifsicle better at producing acceptable results (87% vs. 84%), but not by a statistically relevant amount (p=0.086) and that gifsicle produced fewer broken GIFs (0.71% vs. 1.38%), but again not enough to say it is definitively better (p=0.106). The “better” test found users preferring gifsicle 37% of the time, the prior converter only 16% of the time, but users also preferred one identical image over the other 27% of the time. Again, it is hard to say that gifsicle is better, but it is clear that it is no worse.

    Putting it all together

    The development and testing described above took from late October until the beginning of March. Packaging, deployment, and integration took only a couple of weeks!

    We aren’t done. There is work underway exploring how we handle JPEGs and PNGs. There are a slew of features that we can go after. This was a big step, a necessary step, but not the end for sure. 

    We are a community, it takes a village, there’s no “i” in GIF

    This project couldn’t have happened without the excellent work of Eddie Kohler in creating, maintaining, and enhancing gifsicle. Tumblr’s Site Reliability Engineering group packaged and helped deploy gifsicle onto hundreds and hundreds of machines in our datacenter. Tumblr’s Security Team vetted the code, both by inspection and by attacking it to make sure we stay safe. This was all for the awesome Tumblr creators, but I have to mention qilme/sukme (same dude, two blogs), reallivingartist, and especially gnumblr for their help in understanding and ultimately attacking this monstrous problem.

  • Everything Okay?

    If you or someone you know is suffering from an eating disorder, self harm, or suicidal thoughts, please visit our Counseling & Prevention Resources page for a list of services that may be able to help.

  • We here at Tumblr work hard to ensure you can use your account across a wide range of devices and older systems, but at some point we have to throw our bodies upon the gears and say stop. 

    Specifically, Internet Explorer on Windows XP no longer provides adequate security measures to protect your sensitive online information from being compromised. Consequently, Tumblr will be soon be ceasing support of IE on Windows XP.

    So, if you’re using XP, you’re going to need a different browser. Chrome or Firefox would be our suggestions, personally.

    If you’re absolutely committed to IE, then you’ll have to upgrade to a newer version of Windows.

    We know it’s a tiny bit of a pain to switch, but your security is paramount.
  • We’re experiencing intermittent errors affecting the whole site and are working quickly to resolve it. Thanks for your patience.

  • Hey! Need a link to a specific post?

    Permalinks are permanent links (URLs) to individual posts. You can access a post’s permalink in the Dashboard, as well as on most public blogs.

    From the Dashboard:

    • Hover your mouse over the upper-right corner of any post on the Dashboard.
    • Click the dog ear that appears, or keep hovering to view what time the post was published.

    image

    or:

    • Click the Share icon on the bottom of any post.
    • Click the permalink button that appears. It will open to the post page.

    image

    From a public blog:

    If you are on a public blog, keep in mind that each theme will have its own way of connecting you to a post’s permalink. Look for the icons and links surrounding a post — generally a permalink can be accessed by clicking on a post’s date and time, the link to its notes, or a similar symbol for a hyperlink.

    image

    For instance, hovering over the post time or its notes shows that those both link to the post at:
  • We noticed that you’re using an old version of the Tumblr app. Why? You don’t have to live like this. Get the latest version from the Google Play store or Amazon

    We noticed that you’re using an old version of the Tumblr app. Why? You don’t have to live like this. Get the latest version from the Google Play store or Amazon

  • staff:

Love the new search but wish it looked more like your dashboard? Well, we put a toggle on the results page that does exactly that. Go ahead, try it out. Neat, yes?
“Yes,” you whisper.
Great! Glad you like it.

    staff:

    Love the new search but wish it looked more like your dashboard? Well, we put a toggle on the results page that does exactly that. Go ahead, try it out. Neat, yes?

    “Yes,” you whisper.

    Great! Glad you like it.

  • staff:

    Search!!!!

    Everyone, we’re delighted to show off a feature that we’ve been hard at work on for the last few months and badly addicted to for the last few weeks.

    The new Tumblr Search has been rebuilt, front to back, to help explore all of your 65 billion(!) posts.

    Here are a few of the big additions to check out:

    • You can now use multiple #tags in a single query to find all that #hello kitty #nail art.
    • Searching for “government shutdown” now returns posts tagged #government shutdown or that mention ”government shutdown”. Searching “#government shutdown” will limit the results to posts with that #tag.
    • You’ll now see related blogs in your search results.
    • Use the new Filter menu to filter by post type!

    And in an effort to make the explicit content filter more transparent (and less confusing), we’ve moved the option right to the search results page. We’ve also been able to use this smarter filtering to unblock innocuous keywords in our apps that used to be prohibited by app store policies. #reaction gif #happy!

    Please take the new search for a spin! (And let us know if you catch it acting up.)

    <3

  • The service disruption that affected certain Tumblr users has been resolved. Thanks for your patience!