How to start with Puppeteer and HTTP tracing

921 2

Puppeteer is a Node library which provides a high-Level API to control Chrome over the Dev Tools protocol. Basically, everything you do manually in the browser can be done via Puppeteer. A couple of examples are:

  • Generate screenshots and PDFs of pages.
  • Automate form submissions, UI testing, keyboard input.
  • Create an up-to-date, automated testing environment.
  • Capture a timeline trace to help you find performance issues.
  • Test Chrome extensions.

In this article, I will focus on performance and will give you a simple step by step approach on how to start with Puppeteer and create a short HTTP tracing, which is logging all the web browser interactions.

A couple of weeks ago we started an experiment at client side to check if Puppeteer can help us with performance testing a web application. Where we first use JMeter in combination with our Selenium Page objects we are planning to switch to Puppeteer. The main reason for this is that the Selenium framework is evolved in such a way it’s not that easy anymore to build a jar from the framework and make it work together with JMeter. Reading about Puppeteer made me curious and is apparently very efficient and easy to script.

Before you start

Before we start scripting a couple of preconditions need to be in place:

  • Code editor
  • Install NPM, Node JS, Puppeteer

To start scripting a Code editor needs to be available otherwise we are not able to script and run our code. If you don’t have a code editor installed on your device I will recommend you to install Visual Studio Code. Visual Studio Code is free and available on Linux, MacOs, and Windows.

After installing our Code Editor we are going to install Node Js. This is needed because Puppeteer is a Node library. When installing Node JS we get immediately NPM installed on our device.

Before we are going to install Puppeteer we are first going to create a project directory:

In this project directory we will install Puppeteer with the following command:

When the installation is done we are ready to code!!

Let’s start coding

First of all, let’s create an empty JavaScript file. When this is done we are going to import the puppeteer library by adding the following line:

After this, we can make use of the functions puppeteer is offering. We will be able to start a chrome browser, navigate to a specific page and close the browser again:

Let’s check if this works by starting the little script with the following command in your terminal. Don’t forget to use the file name of the file that you created. In my case this is test.js:

C:\iovio_test>node test.js

If your script runs successfully let’s add some extra steps and do a bit of navigation to the blog section of this web page. Your script will look like the following then:

To test your script you can use the following command in your terminal:

HTTP Tracing

If your script ran successfully it’s time for the next step: we are going to make a HTTP tracing during the execution time of the script. The first action we need to do is to add an extra package with the name puppeteer-har. We can get this package via NPM by the command:

When the package is finished with downloading, we can extend our script with the following code at the top of the script. Your first two lines of code will look like this:

After this is done we will place the code below between the opening of a new page and then set the height and width of the window. In this couple of lines of code we are creating a timestamp for a unique filename and we are starting the tracing:

To stop the HTTP tracing we need to add an extra line of code at the end of the script:

HAR Analyzer

After you run your script, a Har file is created. The format of the file is basically a JSON object with a particular field distribution. For this example, we are going to use an online HAR Analyzer but be careful. A Har file contains sensitive data like all the information you submitted. For this example everything is fine and we can use the HAR Analyzer from google.

Choose your file and open it. All the requests that happen during your recording will be visible in this analysis.

Example of Result HAR Analysis

Summary and Next steps

After following the steps above you are able to

  1. setup your environment;
  2. install the prerequisites to start with coding:
  3. create a simple script that is opening a page in a browser and performs a navigation step.
  4. Starting a HTTP tracing which can be checked by a Har Analyzer.

For the next article that I am going to write, we will go deeper in how to set up the TICK stack in a Docker container and send our performance results to an InfluxDB, that is part of the TICK stack. In this way, we can create a dashboard of the available data.

I hope you had fun reading and trying the examples above. If you have questions don’t hesitate to contact me for some support or to write in the comments below.

Robert Striekwold

Robert Striekwold

Robert is a principal consultant at IOVIO, Quality Assurance Service company. With over 10 year experience in IT he is currently Product Owner for mortgages, leading his team to ensure quality for the biggest finance player in the Netherlands. He previously managed QA projects in the insurance, retail, and telco industries too, not only banking. Born in the Netherlands, he studied Information Engineering in Amsterdam. He started working as a developer but rapidly moved into Quality Assurance, becoming an inspirational, forward-looking and efficient manager. In the years he specialized in test automation, performance testing, service virtualization, validation, monitoring, and agile practices. He is passionate about technology, especially open source. He is proficient on enterprise solutions too. When off from work he is still very passionate and active: being a father of 2 kids, CrossFit training, snowboarding and, no matter the weather, motorbiking with friends.

2 comments

  1. As it appears, Gatling allows you to import HAR files to generate a script. Perhaps you can look into this as well in case you haven’t already thought of this.

    1. Thank for the comment. I didn’t know but will definitely take a look and let you know my experience with Gatling.

Leave a Reply