February 19, 2021

1091 words 6 mins read

On Image Picker: A Flutter Debugging Story

On Image Picker: A Flutter Debugging Story

CareApp uses Flutter to build our mobile app. I’m a fairly vocal supporter of Flutter; it has allowed us to develop a high-quality cross platform app super fast, with just 2 developers. That said, moving fast can mean sometimes things break. This post outlines a problem that unfortunately made its way to production. While the story is specific to us, hopefully the approach is useful to others trying to debug their Flutter apps.

Part 1: The Report

It’s Friday and everything is looking good. Then Jess, our customer support Legend posts this into Slack:

Hey Michael Marner would you know why these won’t load?

Hey Michael Marner would you know why these won’t load?

We had reports that the media attached to some messages shared through CareApp were not appearing for users. As one of CareApp’s primary uses is for sharing media with family, this was an issue.

Part 2: Understanding The Symptoms

Support requests from users usually are not good enough to jump to a quick fix. User’s don’t always know exactly what the problem is, and certainly don’t understand the technical issues that could lead to a problem. So the first step when investigating any report like this is to understand it as best as we can.

The Report: Cannot see media attached to some messages

To go further we needed to inspect the media. To begin, I tried to view the content and, sure enough, I see a placeholder graphic where an image should be. CareApp pre-processes images on the device (resize) before uploading, so this is strange.

Errors like this are often caused by network connectivity issues, but the file was being served just fine. How were these images corrupt?

Next step was to download and inspect one of the files. There were 2 things that immediately jumped out:

  1. No image editor or viewer could open the file
  2. The file was unusually large

As a last resort I tried opening the file in VLC and, BOOM! It wasn’t an image at all, but a video!

I’m not sure how that is even possible

I’m not sure how that is even possible

So we have a video file parading as an image file. This is much more useful, and leads the investigation further.

We store the filename and mimetype for all files uploaded to CareApp in our database. A look in our database verified that the filename of the file was a .jpg, and the mime type we recorded was image/jpg.

So we now had 2 possibilities:

  • The user intended to select a photo, and was somehow able to select a video, or
  • The user intended to select a video, but the video was incorrectly uploaded as an image file

Jess got on the phone to our customer and verified that they had tried to share a video, but it didn’t work. Now we knew where to go to fix this.

Part 3: Jumping into source code

At this point the following facts are relevant:

  • We use the image_picker Flutter plugin for accessing the device gallery
  • The user selected a video from the gallery to share in CareApp
  • While they selected a video file, we stored the file as an image
  • As our API didn’t think the file was a video, our video transcoding pipeline didn’t run

It’s fairly obvious that the issue was somewhere inside image_picker. Running CareApp in a debugger confirmed that even when the user selected a video, the file had a .jpg extension. We could only replicate the issue on some Android devices, which indicated the problem was most likely in the Android implementation of image_picker.

Part 4: Flutter Issues

As a general principle, as soon as you are fairly confident the problem isn’t in your code, you should look to see if there’s a Flutter issue that matches your symptoms. At the time of writing, there were 8,289 of them. A quick search uncovered this issue from March 2020:

image_picker returns .jpg extension when picking Video or GIF

There’s pull request to fix this issue, opened in July 2020 and still open. We are already running our own image picker fork, so we applied the PR to our fork and, sure enough, the bug went away.

We also added some conservative fallback code to ensure that, no matter what, a video wouldn’t be saved as an image:

    final video = await ImagePicker().getVideo(source: source);
    String mime = lookupMimeType(video.path);
    String name = basename(video.path);

    // getVideo should always return a video file, so force a mime type if we get back `.jpg`
    if (!mime.startsWith('video')) {
      mime = 'video/mp4';
      name = setExtension(video.path, 'mp4');
    }

CareApp 2.16.1 released to the Play Store a short time later and the issue was resolved. We manually updated the metadata and triggered our video encoding pipeline for affected media, and the problem went away for our users. Importantly, they didn’t have to do anything - from their point of view the messages just started working.

So I guess to summarise:

  • Learn as much as you can about the issue through data in the app. User reports aren’t good enough
  • Figure out how to replicate the issue
  • Look for existing bugs and pull requests before diving into the code yourself
  • Dive into the code if you need to

Part 5: Flutter Issues

I understand that the Flutter team is fairly small and they’re trying to do a whole lot. Since I’ve been working with it, the mission has crept from wanting to build the best damn cross platform toolkit for mobile devices, to trying to build a universal toolkit for all platforms. This has a cost - and that is the less-sexy maintenance of existing plugins.

But for Flutter to truly be accepted the first party plugins need to be rock solid. They’re starting to solve this issue, which is great to see. Let’s hope the first party plugins see some significant improvements in the coming months. (Hey Google - maybe look beyond Mountain View for developers?)

The bug in question is labeled an annoyance - I reckon it’s a showstopper that makes the plugin unusable without workarounds. The PR has been open for 8 months, and is only un-merged because the (non-Google) author hasn’t had time to write unit tests. Meanwhile, the first party plugin is broken for everyone, without even a warning in the documentation.

Thankfully we identified and worked around the problem quickly. But if you’re evaluating Flutter, factor in plugin bugs into your cost-benefit. Use as few plugins as you can get away with, and pin the versions so you don’t accidentally ship a new problem.

Flutter still wins the cost-benefit analysis, but it ain’t all roses.

comments powered by Disqus