YouTube Comment Language Filter

YouTube has no way to view comments by language. This is the story of building "YouTube Comment Language Filter" through its alpha, beta, and general release.

• anaclumos/youtube-comment-language-filter
• Chrome family installation links (Chrome, Brave, Naver Whale, Microsoft Edge)
• Firefox family installation link

0. The idea

One day, out of frustration and curiosity, I decided to look at how YouTube's comment HTML is put together. After poking at a bunch of values in the console, I wrote a JavaScript file that removes every comment without Korean characters. My past experience with a project called HangulBreak.py about Hangul and Unicode conventions was helpful.

🧼 The first filter script. It removes YouTube comments that don't contain Hangul. It's also stored in the first commit of the GitHub repository.

var commentList = document.getElementsByTagName("ytd-comment-thread-renderer");
var comment;

function containsUnicode(str, startUnicode, endUnicode) {
  for (var i = 0; i < str.length; i++) {
    if (
      startUnicode.charCodeAt(0) <= str.charCodeAt(i) &&
      str.charCodeAt(i) <= endUnicode.charCodeAt(0)
    ) {
      return true;
    }
  }
  return false;
}

for (var x = 0; x < commentList.length; x++) {
  comment =
    commentList[x].childNodes[1].childNodes[1].childNodes[3].childNodes[3]
      .innerText;
  if (containsUnicode(comment, "가", "힣")) {
    // comment = "한글임 \n" + comment;
  } else {
    // console.log(typeof commentList[x]);
    commentList[x].parentNode.removeChild(commentList[x]);
  }
}

for (var x = 0; x < commentList.length; x++) {
  console.log(
    commentList[x].childNodes[1].childNodes[1].childNodes[3].childNodes[1]
      .innerText,
  );
  // The author's name and date of creation are concatenated together. "Name\nDate Created"

  console.log(
    commentList[x].childNodes[1].childNodes[1].childNodes[3].childNodes[3]
      .innerText,
  );
  // Comment!
}
// End of code

/* Notes: This is initial code, and while it works in some cases.
 * there are a number of errors and performance issues.
 * The code, which has been debugged and improved, is uploaded to GitHub.
 */

At this point, I thought I could grow this code into a project. However, there were a few problems with this script.

• First, I had to paste the code from the console, which made it difficult and inconvenient for others to use.
• It didn't rerun on its own, so I had to re-run it in the console whenever the comments reloaded.
• Since it could only filter out Korean characters, it was very limited in use, and it was slow to run.

Fixing these became the obvious next goal. A Chrome extension seemed like the easiest place to start.

• Edge and Whale are also Chromium-based, and Firefox add-ons can be created as Chrome extensions.
• Safari doesn't play YouTube 4K videos, so many Safari users also use Chromium to view YouTube.
• Internet Explorer has lost support for YouTube.

I also wanted people who aren't great with computers to use it easily, so it had to sit between the video and the comments, "just like YouTube does," where you'd naturally move your cursor from the video down to the comments. So I didn't want a Chrome extension popup. I wanted the menu built right into the YouTube page.

Now the goal was concrete.

• This Chrome extension would insert a language control interface between the video and the comments.
• The interface will allow all comments to be filtered automatically. It will continue to work even when more comments are loaded.
• It should be scalable so that adding new languages is not a major challenge.
• It should be reasonably fast.

See also: versioning

Everyone numbers versions differently. In this post I'll use the following.

• First digit is a major update.
• First decimal is a feature, second decimal is a bug fix.
• Third decimal is a store re-upload with no code change.

1. Alpha versions should be released as soon as possible

An alpha (or more precisely, a Minimum Viable Product) needs at least the basics working. Specifically, it had to

1. refilter automatically on every new load,
2. let you see the full comments again anytime, and
3. install in one click.

The first question was, "How do we make it happen automatically?".

Method 1. Time-based autoplay

For people who just turn on a video and watch it, rerunning the filter on a timer wastes performance. But if the gap between runs is too long, reading comments gets annoying. It was the first option I thought of, and the first one I threw out.

Method 2. Detect YouTube's loading icon

YouTube's loading icon

When I checked, the <can-show-more> tag appeared and disappeared under the comment whenever the loading icon appeared, so I thought I could catch the moment when YouTube's obfuscated javascript code inserted the <can-show-more> tag and re-run the filter accordingly.

Method 3. MutationObserver

Then I discovered MutationObserver in JavaScript. The idea is to set a target to observe and a config to observe, and then run a callback function when a change occurs that meets the config. I used the YouTube comment HTML as the target and made it react to changes in the childNodes and attributes in the HTML. Just as we wanted, it was re-run every time the comment loaded. Problem solved.

Performance tanked, though. console.log was firing tens of thousands of times, so I deleted it and things got usable. To speed things up a bit, instead of testing the language of the comment each time, I wrapped the comment with display:none; and recorded the result in an HTML tag. When I clicked the button to view the full comment, it would remove the display:none;, and when I ran it again, it would only use the tags I had recorded. Solution 2. (I didn't realize this until later, but it wasn't a complete solution. This way of utilizing annotations didn't improve speed much and created more problems. See YouTube's SPA-ness during beta #2.)

Next three days went to learning how Chrome extensions are structured and porting the code over, mostly from the official docs, Stack Overflow, and a Udemy course. Solution 3.

Finish the alpha version

We started development on March 6, 2020, and finished the first alpha version on March 11. We shared the alpha with a small group of testers and spent about a week collecting user reviews and researching improvements and bugs.

2. Beta version

We found and improved several problems in the alpha version.

① YouTube's Lazy Loading Issue

YouTube loads content lazily. Lazy loading refers to the practice of waiting and fetching information when it's needed, rather than loading everything at once. Lazy loading is usually applied to large images, but YouTube has taken it a step further and lazily loads the HTML itself! (I don't know if that's accurate, but that's what I saw at the time of development).

When you first land on a video page, the HTML for the comment interface doesn't exist. When the user starts scrolling to view the comments, a gray loading icon appears and loads the comment HTML. (Why? Because it saves querying the comment DB. Seems like a clever solution.) Since the comment HTML doesn't exist at the document_end of the extension, the extension throws an error and exits shortly after.

Solution.

Lazy loading kept me from seeing the comments themselves, but I could still stick the filter menu into the comment box and it worked fine. Luckily the comment box showed up soon after the page loaded (at document_end), so if the comment HTML wasn't there yet, I had it rescan for the comment box Xpath every 0.5 seconds.

This may seem a bit puzzling, as I had previously ruled out retrying at regular intervals as a waste of computation. In the previous case, the timer would loop indefinitely because it was re-checking the comments after a certain amount of time, whereas in this case, it only retries until the comments window loads. In practice, this workaround terminates normally after one or two retries. If it takes longer than that, your internet is too slow to view YouTube itself.

Then came the follow-up bugs. Because I wasn't very good with JavaScript, I didn't know how to efficiently retry certain functions after a certain amount of time. YouTube would sometimes load infinitely when accessed by video URL or directly from search results. It turned out that we needed to make the JS retry after a certain amount of time, but by making it wait for a certain amount of time, we were preventing other JS from running. This bug can also be found in GitHub Issue #4 and was finally fixed in v1.1.4. (The bug showed up about once in 20 times, and it was harder to reproduce the error situation than to find the bug in the error situation.)

② YouTube's HTML component recycling issue

For a Chrome extension to get into a website and change things, you have to list every address it'll touch in manifest.json. Initially, I used https://*.youtube.com/watch* because I thought it would only need to run on the video page. However, I ran into a problem when I tried to access the video from the YouTube home screen.

YouTube reuses a lot of components from HTML. If you've ever watched a YouTube video and pressed the i button to launch the miniplayer, you'll recognize this. YouTube doesn't actually take you to the video page, it just (1) covers the existing screen, (2) puts a new page on top, and (3) replaces the address in the web address bar. So naturally, when you press the i button to open the miniplayer, you'll see the same window you saw before playing the video.

Widening the pattern to https://*.youtube.com/* made it worse. Now the menu got injected in wrong places, even on pages with no comment box.

Also, on video pages, even if the filter menu was inserted correctly, if you navigated to another video with the filter enabled, it would sometimes filter the "current video's comments" based on the "previous video's comment language". This also seemed to be caused by YouTube's recycling of the comment component. I mentioned earlier that I was jotting down the results in the comment HTML to speed things up, but this also scrambled the results and made the error bigger.

Solution

The extension watches which site you're on. If the domain is YouTube, it gets ready to add the filter, then checks the domain each time by hand and only adds the filter on youtube.com/watch.

In this case, we need the "Can read user's history" permission when installing the extension. For this reason, I've included a note about the permissions used in the corresponding commit and installation completion page. As you can see from the GitHub open source, the browsing history is not sent externally.

I also set it up so that when you navigate to a new page with filters on, it will (1) reset all filters and filter results, (2) reload the comments on the new page, and (3) wait.

Sometimes the problem would reappear. However, this seems to be more of a YouTube bug. Even without this extension, YouTube's web commenting system is notoriously buggy, with incorrect comments appearing and comments on videos getting mixed up. In this case, a refresh should fix it.

③ Speed issues

In the alpha the filter was really slow, and just deleting every console.log sped it up a lot. As long as the filter runs faster than people read comments, it's usable, so fixing the problems above came first. Later I found this problem too.

for (var comment of commentList) {
  if (comment.id === "") {
    var commentString =
      comment.childNodes[1].childNodes[1].childNodes[1].childNodes[3]
        .childNodes[3].childNodes[1].innerText;
    if (containsSelectedLang(commentString)) {
      comment.id = "contains-SelectedLang";
    } else {
      comment.id = "no-SelectedLang";
    }
  }
  if (comment.id === "no-SelectedLang") {
    comment.style = "display: none";
  }
}

The comment address in commentString doesn't currently work because it has changed several times with YouTube's updates.

Solution...?

We still haven't fully fixed it. For now, in v1.2 we switched to script names instead of language names, to make it clearer that this thing tells scripts apart. The plan is to add a natural language processing module in the end. But Chrome extension policy means using one would take tearing up a good chunk of the code, so I'm still looking into it.

3. Full version

The full version had no speed or feature improvements to the filter itself. It was all usability. A settings window to hide unused languages, a landing page that opens right after install, a short bug-report guide, and a redone promo image.

I did a small experiment a while back, and the difference between the number of people who came in with and without the promo image was about 4x.

Finally, I ported the Chrome extension to Firefox as an add-on. Write-up in Porting a Chrome Extension to Firefox Add-on (original post).

Overall thoughts on v1

My goal in releasing v1 was to create a deliverable that I could use without thinking about it. If an extension breaks along the way and you start to care about it, it ruins the fun of watching YouTube.

But the current v1 doesn't break the fun. I can just use it without thinking about it and it works fine. (It just all... works!)

In addition, the original purpose of finding Korean comments works almost flawlessly. After all, Korean comments will have at least one Hangul character in them.

In the future

I'm working on v2. The most popular feedback on v1 was "more styles" and "better language detection". A fundamental problem with Chrome extensions is that it is very complicated to add external files or modules from within the Chrome extension itself. Using the npm library is also cumbersome. The solution is to use something called Webpack, which requires a significant rewrite of the code. Still, I can think of a lot of fun ways to use it, so I'm working on v2 in my spare time.

There's also talk that YouTube is experimenting with the percentage of foreigners viewing Korean videos by deliberately pushing Korean comments down in the comment rankings, so I don't think this project will ever end.

That's why the title says Part 1. Someday, if v2 is completed, or if something happens that deserves to be remembered, you'll see part 2 of this post.