Build a Chrome extension to automate a part of your routine

In this post I talk about:

Share the Post:

If you are using Google Chrome, installing extensions is often a good way to improve your workflow and work more efficiently. To name just one, I really love the Robots exclusion checker which enables you to visually check if a page is indexable when you browse a website. Before using it, checking the canonical tag, the meta robots, the robots.txt file & the HTTP header was quite time-consuming, even if we obviously have tools (like Screaming Frog) to speed up the process.  

Great, but what if your need is not covered by an existing extension? In today’s post, I want to explain how to create & publish a Chrome extension from scratch from a newbie (myself) point of view.

The idea 

As most SEO professionals, I’m often using Google Search Console for my projects. When it comes to keywords data and even if the query report is kind of sampled, the tool gives us the most comprehensive data we can find. Nevertheless, when I’m performing keyword research for new content ideas or to improve my existing contents, the typical process is the following:

  • Analyze data from GSC UI or using the API 
  • Extract search volume using a third party tool (Google Ads, Semrush….)  
  • Merge everything using Excel or Sheets

But isn’t there a way of displaying the search volume directly within the GSC API? It would cut down the time spent on this task significantly because all data would be unified in the same UI. 

When I realized that no other extension (that I knew of) offered this functionality, I decided to build my own. 

How does a Chrome extension work? 

If you don’t know it yet, Chrome extensions rely heavily on JavaScript. Based on the complexity of the one you want to create, you may need several files to make work, but you need at least: 

  1. manifest.json: a file including several key information on the extension, like its name, its description and when it should be executed once installed. For more information, please see the official documentation
  2. code.js: the name can be different (you specify it in your manifest.json file), but this JavaScript file includes the code to be executed by your extension. In my case, all the instructions are there. 

If you are curious to know how your favorite extensions work, you can follow this tutorial. 

How does my extension work? 

If you use the extension, you can do two things: 

  • Display search volume directly in GSC UI 
  • Add a new download button to download these volumes (if you want) along with your keywords data 

The full code is available in my GitHub repository, but let me explain how it works piece by piece.

JavaScript vs Python 

If you’ve read other articles I’ve published on my blog, you may know that I’m regularly using Python to speed-up my work. In other words: JavaScript is not my thing. Therefore, how the heck did I create an extension based on this language?

  • Collaboration: the SEO industry is great. If you ask for help, there is always someone willing to help you. In my case, I spoke with Jose Luis Hernando who’s more than knowledgeable on JS and he helped me a lot through the project. I also asked Fede Gomez to solve a specific issue I had at the end of the project, which I will explain later. 
  • Knowledge transfer: even if JavaScript has some key differences with Python, some concepts are basically the same. You just have to Google quickly how to write a loop, but you don’t have to understand the logic because you already learned it before. 

The Logic 

The logic followed by the extension is pretty simple:

  1. Download keywords from GSC UI
  2. Extract their volumes from an API 
  3. Add them into the UI 

Download keywords from GSC 

function get_keywords() {
  //array where we will store our keyword
  let arr = [];
  //gets table
  //there are several tables with the same class
  var oTable = document.getElementsByClassName('i3WFpf')[0];
  //gets rows of table
  var rowLength = oTable.rows.length;
  //loops through rows
  for (i = 0; i < rowLength; i++) {
    //gets cells of current row
    var oCells = oTable.rows.item(i).cells;
    //loops through each cell in current row{
    arr.push(oCells.item(0).innerText);
  }
  // Remove "Top Queries" header from array
  arr.shift();
  return arr;
}

The data we want to extract is included in a table whose class is “i3WFpf”. I really hope that Google won’t change that soon, otherwise my extension will need to be updated!

To extract the keywords, I loop through this table and extract the first row (the keyword). I end up with an array (a list) of up to 1.000 keywords, which is GSC’s UI limit. This step was pretty easy because I honestly believed that the source code was going to be messier, like in Amazon for instance. Fortunately, that was not the case at all.

Extract their volumes from an API

Before even explaining the code, I want to highlight which API I used and why. My extension uses Keyword Surfer’s (that you may know for their Keyword Surfer extension) because:

  • It is free
  • It doesn’t require neither authentification nor API key (at least now)
  • It allows bulk request: with one request, you can retrieve the volume for up to 50 keywords.

The last advantage was decisive, because having to execute up to 1.000 requests (for up to 1.000 keywords) would have impacted A LOT the speed at which the extension can retrieve search volumes. It also means that I won’t surcharge their API with my extension, which is the last thing I want.

That being said, the API is far from perfect:

  • Some volumes are not updated very often
  • Low-volume keywords are badly reported
  • As usual, keywords are grouped. “Buy car” and “buy cars” will return the same volume (or one of them will return 0) when we do have separate metrics in GSC. This is a very known limitation for most API though, not only Keyword Surfer’s.
  • If the Keyword Surfer doesn’t have data for a specific keyword, it doesn’t return 0 but noting. Something we had to take into account in our logic.

That being said, let’s have a look at the code:

async function get_search_vol(chunk, url) {
  const requestUrl = url; // URL to request
  const response = await fetch(requestUrl); // Make request to Keyword Surfer
  const json = await response.json(); // Transform response to JSON
  //loop the response and return an array with volumes
  let keywords = {};
  for (i = 0; i < chunk.length; i++) {
    keywords[chunk[i]] = json[chunk[i]]?.search_volume ?? 0; // If keyword has data get the data else return 0 (optional chaining operator (?.) + Nullish coalescing operator (??))
  }
  return keywords;
}

//function to divide an array in X arrays
//Source: https://ourcodeworld.com/articles/read/278/how-to-split-an-array-into-chunks-of-the-same-size-easily-in-javascript
function chunkArray(myArray, chunk_size) {
  var index = 0;
  var arrayLength = myArray.length;
  var tempArray = [];

  for (index = 0; index < arrayLength; index += chunk_size) {
    var myChunk = myArray.slice(index, index + chunk_size);
    // Do something if you want with the group
    var finalChunk = [];
    for (y = 0; y < myChunk.length; y++) {
      finalChunk.push(myChunk[y].replace('"', '').replace('"', '').replace('&',''));
    }
    tempArray.push(finalChunk);
  }

  return tempArray;
}

//generate the Keyword Surfer's URLs for our chunks
function generate_urls(chunks, country) {
  //output
  const arr = [];

  //Base URL to generate our list of urls
  const base_url = `https://db2.keywordsur.fr/keyword_surfer_keywords?country=${country}&keywords=[%22`;

  // Loop through chunk to create array of keywords in request
  for (i = 0; i < chunks.length; i++) {
    var url = base_url.concat(chunks[i].join('%22,%22'), '%22]');
    arr.push(url);
  }
  return arr;
}

To extract volumes, there are several steps:

  • A function called chunkArray to divide our list of keywords in chunks of X elements. We need to perform that action because we can request up to 50 keywords to Keyword Surfer’s API.
  • A function called generate_urls to create a list of requests URLs using our chunks. The structure is pretty simple: https://db2.keywordsur.fr/keyword_surfer_keywords?country=us&keywords=[%22buy%20car%22,%22buy%20cars%22] (example with 2 keywords), even if special characters (see this reported issue in GitHub) can break the API and are currently the main reason why users can’t use the extension properly sometimes.
  • A function called get_search_vol to retrieve the search volume. Indeed, Keyword Surfer’s API provides us with more than just the search volume, we therefore need to process the result to extract only the information we need.

We then add these functions altogether inside another one to obtain the expected output: a list of keywords (from GSC) with their associated volume (from Keyword Surfer’s API).


async function getData(country) {
  const kws = get_keywords(); // Get all keywords from GSC
  const chunks = chunkArray(kws, 50); // transform keyword in multiple arrays of 50 keywords
  const urls = generate_urls(chunks, country); // Build Request URL
  const allKeywords = {}; // Store future reponses in hashmap

  // Loop through GSC set of keywords and request keywoFrd surfer data
  for (let i = 0; i < urls.length; i++) {
    var sv = await get_search_vol(chunks[i], urls[i]);
    var keys = Object.keys(sv);
    for (let y = 0; y < keys.length; y++) {
      allKeywords[Object.keys(sv)[y]] = sv[Object.keys(sv)[y]];
    }
  }
  // console.log(allKeywords); // Just to check the output
  return allKeywords;
}

Amazing! The only this we now need is to display this information to the user!

Add them to the UI

function createCell(text) {
  var cell = document.createElement('td');
  var cellText = document.createTextNode(text);
  cell.appendChild(cellText);
  cell.setAttribute(
    'style',
    'font-size:12px;font-weight:bold;text-align:center;padding:18px 28px;'
  );
  return cell;
}

async function addVolumes(country) {
  const volumes = await getData(country); // Wait to get hasmap of search volumes
  const tbl = document.getElementsByClassName('i3WFpf')[0]; // Select table
  // Future CSV
  let csvExport = '';

  // Loop through rows
  for (let i in tbl.rows) {
    // For some reason there is an undefined row at the end
    if (i === 'length') {
      break;
    } else {
      // Select each row
      let row = tbl.rows[i];
      // Create line for future CSV
      const line = [];
      // Select first cell (query)
      const query = row.cells[0].textContent;
      // Add header
      if (query === tbl.rows[0].cells[0].textContent) {
        row.appendChild(createCell('Search Volumes'));
      } else {
        // If there is search volume data add it
        if (volumes[query]) row.appendChild(createCell(volumes[query]));
        // If not add 0 search volume
        else row.appendChild(createCell(0));
      }
      // Loop through cells
      for (const cell of row.cells) {
        const text = cell.textContent;
        line.push(text.replace(/,/g, ''));
      }
      // Create each filled line for future CSV
      csvExport = csvExport.concat(line.join(','), '\n');
    }
  }
  createDownloadButton(csvExport);
}

The logic is pretty simple here:

  • A function called createCell is used to create cells in our GSC table. The CSS added (inline) in this function can be seen in your source code once the volumes have been retrieved by the extension.
  • A function called addVolumes add the volume column to our entire table. If the keyword has no data in Keyword Surfer (as mentioned above), we add 0 by default.

What about user input?

If you used my extension, you remember that you need to specify your country to use it.

Storing an user input to be reused later on was actually the most difficult part of the whole process. The code may look simple but:

  • The documention is not very clear and understanding why your code doesn’t work is not easy either, because it often doesn’t trigger any error 🙁
  • Google Chrome has its own standards instead of using JavaScript’s

That being said, the following code allows my extension to save the user input, to be later on used by the main code to retrieve search volume for a given country. In my example, it would retrieve volume for France.

const optionsStatus = document.getElementById('status');
const countryList = document.getElementById('country');
const optionsSaveBtn = document.getElementById('save');

function save_options() {
  var country = document.getElementById('country').value;
  chrome.storage.local.set({ country: country }, function () {
    // Update status to let user know options were saved.
    var status = document.getElementById('status');
    status.textContent = 'Options saved.';
    setTimeout(function () {
      status.textContent = '';
    }, 750);
  });
}

// Restores select box and checkbox state using the preferences
// stored in chrome.storage.
function restore_options() {
  // Use default value color = 'red' and likesColor = true.
  chrome.storage.local.get(
    {
      country: 'fr',
    },
    function (items) {
      document.getElementById('country').value = items.country;
    }
  );
}
document.addEventListener('DOMContentLoaded', restore_options);
document.getElementById('save').addEventListener('click', save_options);

How to debug an extension?

When you create a JavaScript code to be used by an extension, you need to test & debut it first. You have two options:

  • Use the code directly in Chrome: you can run directly JavaScript using the Console tab of Chrome DevTools. Pretty handy at the beginning (because you don’t have to install anything) but it forces you to copy-paste your code quite often.
  • Install your extension: you can install an unverified extension (follow these steps) and use it as a regular user would. It requires you to build the manifest.json though, something you may not want to create at the beginning of the process.

Maybe there are most efficient way to debug an extension, but as a newbie I only used these two, and it was more than enough for my project.

How to submit your extension?

If you want to publish your extension, you can follow the official documentation which explains it pretty well. To summarize the process, you’ll have:

  • To create a developer account and pay the 5$ fee (even if you extension is free)
  • Submit your extension by providing the required information, like its purpose. Please note that you need to justify the permission required by your extension. I actually got my second submission rejected because I required too many permissions when it was not necessary. So be careful with that, because it will slow the full process.
  • Don’t include the word “Google” in your extension name (or any other brand name). My first rejection was caused by this rookie mistake, Google considering that you pretend to be someone else (Google in my case) by using its name.

Apart from these recommandations, the validation process is rather quick (less than a week). If you want to update your extension, the process is even quicker as you obviously don’t need to fill everything again.

Conclusion

Improving my workflow is always one of my priority and despite coming from a Python background, developing this extension was not as hard as I thought. I wouldn’t have achieved it without the help & advice of the persons I mentioned at the beginning of the article though, let’s be honest here 🙂 Extensions are easy to create and can save us a lot of time, so that’s definitely something I’ll be looking at if you know a bit of JavaScript.

Python is popular right now, but we can’t deny that JavaScript has its advantages.

Share the Post: