Hi Friends! Yesterday I found out that the Brotli compression is not enabled. I saw this information on this blog’s W3 Total Cache plugin. Since I’m not familiar with it, I decided to research about it.

What Is Brotli?

Straight from its Github page:

Brotli is a generic-purpose lossless compression algorithm that compresses data using a combination of a modern variant of the LZ77 algorithm, Huffman coding and 2nd order context modeling, with a compression ratio comparable to the best currently available general-purpose compression methods. It is similar in speed with deflate but offers more dense compression.

The specification of the Brotli Compressed Data Format is defined in RFC 7932.

To be honest, I don’t understand most of the above sentence. I only know lossless compression (contrast it against lossy compression, e.g. JPEG). Anyway, that’s another research in another day.

Brotli vs Gunzip

I have configured this blog to use gunzip. Is there further benefit of using Brotli? Let’s read Nick’s conclusion:

For 5 paragraphs of lorem ipsum, Brotli beats gzip by 5%. If I run the same experiment with the front page of reddit.com from 10/01/2015, Brotli beats gzip by 22%! Note that both measurements were using the compressors out of the box without any tweaking of configuration values.

This Blog’s Configuration

This blog’s OS:

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 18.04.6 LTS
Release:        18.04
Codename:       bionic

PHP version:

$ php --version
PHP 7.4.24 (cli) (built: Sep 23 2021 21:35:51) ( NTS )
Copyright (c) The PHP Group
Zend Engine v3.4.0, Copyright (c) Zend Technologies
    with Zend OPcache v7.4.24, Copyright (c), by Zend Technologies

NGINX version:

$ nginx -v
nginx version: nginx/1.17.3

Next steps is where we install and configure all the required components in order to enable Brotli compression.

Install Brotli

Installation command:

sudo apt-get install brotli

Command to verify brotli installation:

$ brotli --version
brotli 1.0.4

Now brotli is installed in the system. Let’s continue by working on PHP installation.

Install Brotli Extension for PHP

First we need to install PHP-dev (so we can run phpize):

sudo apt-get install php7.4-dev

Then run the following commands to get the extension’s source code, build and install:

cd /tmp
git clone –recursive –depth=1 https://github.com/kjdev/php-ext-brotli.git
cd php-ext-brotli
phpize
./configure
make
make install

Edit php.ini, add the following (find the location of the commented extension. e.g: ;extension=xxxx.so):

extension=brotli.so

Reload PHP:

sudo service php7.4-fpm reload
sudo service php7.4-fpm restart

Install Brotli extension for NGINX

Run the following command:

sudo apt-add-repository -y ppa:hda-me/nginx-stable
sudo apt-get update
sudo apt-get install brotli nginx nginx-module-brotli

Edit nginx.conf to load the modules:

        ## ngx_brotli filter module - used to compress responses on-the-fly.
        load_module modules/ngx_http_brotli_filter_module.so;
        ## ngx_brotli static module - used to serve pre-compressed files.
        ## Both ngx_brotli modules could be used separately, but part of nginx-module-brotli package
        load_module modules/ngx_http_brotli_static_module.so;

Further brotli configuration will be handled by W3 Total Cache. Restart NGINX:

sudo service nginx restart

Enable Brotli in W3 Total Cache

Go to plugin settings, Browser Cache:

We need to enable it in three locations: General, CSS & JS, and HTML & XML. Once we applied the settings, verify the .conf generated in /etc/nginx/available-sites:

# BEGIN W3TC Browser Cache
brotli on;
brotli_types text/css text/x-component application/x-javascript application/javascript text/javascript text/x-js text/richtext text/plain text/xsd text/xsl text/xml image/bmp application/java application/msword application/vnd.ms-fontobject application/x-msdownload image/x-icon application/json application/vnd.ms-access video/webm application/vnd.ms-project application/x-font-otf application/vnd.ms-opentype application/vnd.oasis.opendocument.database application/vnd.oasis.opendocument.chart application/vnd.oasis.opendocument.formula application/vnd.oasis.opendocument.graphics application/vnd.oasis.opendocument.spreadsheet application/vnd.oasis.opendocument.text audio/ogg application/pdf application/vnd.ms-powerpoint image/svg+xml application/x-shockwave-flash image/tiff application/x-font-ttf audio/wav application/vnd.ms-write application/font-woff application/font-woff2 application/vnd.ms-excel;
gzip on;
gzip_types text/css text/x-component application/x-javascript application/javascript text/javascript text/x-js text/richtext text/plain text/xsd text/xsl text/xml image/bmp application/java application/msword application/vnd.ms-fontobject application/x-msdownload image/x-icon application/json application/vnd.ms-access video/webm application/vnd.ms-project application/x-font-otf application/vnd.ms-opentype application/vnd.oasis.opendocument.database application/vnd.oasis.opendocument.chart application/vnd.oasis.opendocument.formula application/vnd.oasis.opendocument.graphics application/vnd.oasis.opendocument.spreadsheet application/vnd.oasis.opendocument.text audio/ogg application/pdf application/vnd.ms-powerpoint image/svg+xml application/x-shockwave-flash image/tiff application/x-font-ttf audio/wav application/vnd.ms-write application/font-woff application/font-woff2 application/vnd.ms-excel;
location ~ (robots\.txt|[a-z0-9_\-]*sitemap[a-z0-9_\-]*\.(xml|xsl|html)(\.gz)?) {
    try_files $uri $uri/ /index.php?$args;
}

Finally, verify the result in browser:

That’s all friends! I hope it helps.

This post will be impossible without information from the following sources:

1. https://github.com/kjdev/php-ext-brotli
2. https://nixcp.com/brotli-compression-nginx/
3. https://computingforgeeks.com/how-to-enable-gzip-brotli-compression-for-nginx-on-linux/

I never professionally dealing with WCF in my working life. All my previous projects were not using WCF. By using, I mean providing services for other systems to consume. If any, they were only consuming web services from other systems. So WCF is one of many areas in .NET-world which I am yet to explore deeply.

But why WCF, and why now? I myself consider this decision to pick up WCF now is quite silly. First, that WCF is no longer actively develop by Microsoft. Second, Microsoft is already working on the replacement (WCF Core). But since it’s not yet production-ready, Microsoft is currently recommending gRPC for production.

But in Singapore, many businesses (especially Government-linked) are fully Microsoft-shop and heavily using WCF. That is my conclusion after seeing many job postings ask for WCF skills. So this is my effort to improve my professional skill, and widen my possible career-net 😀

In order to be able to host WCF projects in my laptop (Windows 10 upgraded to Windows 11), I need to enable IIS. I did stumbled here and there in my progress, so I write it down here for my own future reference. And hopefully, it will also help others. So here’s what I’ve learned.

How To Enable IIS

1. Press Window button
2. Type turn Windows features on or off
3. Select and expand “Internet Information Services”
   
4. Select and expand “World Wide Web Service”
   
5. Select and expand “Application Development Feature”. Select the .NET version to work with.
   
6. Expand “.NET Framework 4.8 Advanced Services”
   
7. Select and expand “WCF Services”
   

Now we need to verify that IIS is able to serve .svc files (Web Service Descriptor, shown below).

<%@ ServiceHost Language="C#" Debug="true" 
    Service="Service" 
    CodeBehind="~/App_Code/Service.cs" %>

1. Press Window + R keys
2. Type inetmgr, then press Enter
3. Click “Handler Mappings”
   
4. Verify that *.svc handlers are listed.
   

That’s all friends. I hope it helps. Cheers!

After getting familiar with Locust, it’s time for me to try to write the real test. The project to be tested is using ADFS authentication. So if an unauthenticated user visited its URL https://project.mycompany.com, it will be redirected to:

https://fs.mycompany.com/adfs/ls/?wtrealm=https%3A%2F%2Fproject.mycompany.com&wctx=WsFedOwinState%3__SUPER_LONG_BASE64_TEXT__&wa=wsignin1.0

Which prompts the username and password:

Once I entered the correct username and password, the project’s landing page is shown.

To help me to know the submission sequence, I logout, open Chrome’s DevTools, then repeat the login process. Here’s what I see on DevTools -> Network -> Doc tab:

We can see that there was 3 redirects (302):

1. When we load https://project.mycompany.com
2. When we submit the username and password (POST)
3. POST to https://project.mycompany.com

First, let’s observe the form of the login page:

<form method="post" id="loginForm" autocomplete="off" novalidate="novalidate" 
	onkeypress="if (event &amp;&amp; event.keyCode == 13) Login.submitLoginRequest();" 
	action="/adfs/ls/?wtrealm=https%3A%2F%2Fproject.mycompany.com&amp;wctx=WsFedOwinState%3D__LONG_BASE64_TEXT__&amp;wa=wsignin1.0&amp;client-request-id=__GUID_VALUE__">
	<div id="error" class="fieldMargin error smallText" style="display: none;">
		<span id="errorText" for=""></span>
	</div>

	<div id="formsAuthenticationArea">
		<div id="userNameArea">
			<label id="userNameInputLabel" for="userNameInput" class="hidden">User Account</label>
			<input id="userNameInput" name="UserName" type="email" value="" tabindex="1" class="text fullWidth" spellcheck="false" 
			placeholder="someone@example.com" autocomplete="off">
		</div>

		<div id="passwordArea">
			<label id="passwordInputLabel" for="passwordInput" class="hidden">Password</label>
			<input id="passwordInput" name="Password" type="password" tabindex="2" class="text fullWidth" placeholder="Password" autocomplete="off">
		</div>
		<div id="kmsiArea" style="display:''">
			<input type="checkbox" name="Kmsi" id="kmsiInput" value="true" tabindex="3">
			<label for="kmsiInput">Keep me signed in</label>
		</div>
		<div id="submissionArea" class="submitMargin">
			<span id="submitButton" class="submit" tabindex="4" role="button" 
			onkeypress="if (event &amp;&amp; event.keyCode == 32) Login.submitLoginRequest();" 
			onclick="return Login.submitLoginRequest();">Sign in</span>
		</div>
	</div>
	<input id="optionForms" type="hidden" name="AuthMethod" value="FormsAuthentication">
</form>

From above, we can confirm that we need to extract out the form’s action value.

Looking at the first POST (submit username and password), we can confirm the parameters submitted.

After submit, it redirected to a page:

<html>
	<head>
		<title>Working...</title>
	</head>
	<body>
		<form method="POST" name="hiddenform" action="https://project.mycompany.com:443/">
			<input type="hidden" name="wa" value="wsignin1.0"/>
			<input type="hidden" name="wresult" value="&lt;t:RequestSecurityTokenResponse ....VERY LONG ENCODED XML... &lt;/t:RequestSecurityTokenResponse>"/>
			<input type="hidden" name="wctx" value="WsFedOwinState=...QUITE LONG BASE64 TEXT ..."/>
			<noscript>
				<p>Script is disabled. Click Submit to continue.</p>
				<input type="submit" value="Submit"/>
			</noscript>
		</form>
		<script language="javascript">window.setTimeout('document.forms[0].submit()', 0);</script>
	</body>
</html>

We can see above, it is an auto-submit form where the ADFS authentication result is submitted.

Base on the sequence above, here’s how the construct of our test:

from locust import HttpUser, between, task
import logging

#to parse HTML
from bs4 import BeautifulSoup

class WebsiteUser(HttpUser):
    host = "https://project.mycompany.com"
    #wait for 3 to 10 seconds before each request
    wait_time = between(3,10)

    def on_start(self):
        resp = self.client.get("/", allow_redirects=True) 
        soup = BeautifulSoup(resp.text, "html.parser")
        logging.info(soup.title)
        
        #steps to handle unauthenticated users
        if soup.title.contents[0] == "Sign In":
            # find the login form's action
            logging.info('Trying to login')
            action = soup.form["action"]
            url = "https://fs.mycompany.com" + action
            logging.info("Post URL: " + url)
        
            #submit login information
            loginResp = self.client.post(url, {"UserName":"__USERNAME__", "Password":"__PASSWORD__", "AuthMethod": "FormsAuthentication"}, allow_redirects=True)            
            logging.info("Login Result URL: " + loginResp.url)
            #parse the login submit result
            soupLogin = BeautifulSoup(loginResp.text, "html.parser")

            #construct the auto-submit form
            inputs = soupLogin.find_all("input")
            dict = {}
            for inp in inputs:
                if inp.get("name") != None:
                    dict[inp["name"]] = inp["value"]

            # POST auto-submit form
            resp = self.client.post("/", dict, allow_redirects=True) 
            soup = BeautifulSoup(resp.text, "html.parser")
            # the title should be project's landing page title
            logging.info(soup.title)

    @task
    def index(self):
        resp = self.client.get("/", allow_redirects=True)
        soup = BeautifulSoup(resp.text, "html.parser")
        logging.info("URL: " + resp.url)
        logging.info(soup.title)

With that, we should be able to write more @task to test the performance of each pages. Hopefully, we can identify the choke-points before opening access to the users.

That’s all friends! I hope it helps. Cheers!