I am planning to get a new laptop. And its RAM must be at least 32 GB. So I visited some prominent e-commerce websites which are based in Singapore. To my frustration, none of these websites allows me to filter the laptops based on the RAM size.

My first immediate instinct was telling me to scrape the data from these websites, compile then share it here using JSON Table Editor. But I have a small worry that this kind of activity is deemed illegal in Singapore. So I googled around, and found this enlightening quora answer by Bernard Chung. Bernard says that there are four laws that could apply to web scraping activity:

1. Contract Law
2. Tort Law
3. Intellectual Property Law
4. the Computer Misuse and Cybersecurity Act

So, which of the law above that could expose me to legal problem should I carry on with web scraping? I’m not planning to use any illegal means to access their website (e.g. exploit vulnerabilities, SQL-Injection, hacking/cracking), so law no 4 should not be relevant to this. What about Intellectual Property Law? Since I am only after the data, not other things that might classify as their Intellectual Property, law no 3 should not be relevant. What about Tort Law? Since I will access their website just like a normal human do (albeit much much faster), I should not be considered as trespassing.

So it’s only left to Contract Law. If these websites put out a Terms and Conditions (TOC) or Term of Service (TOS) which specifically forbid automated data extraction, my action will now be considered as illegal.

I guess now I need to find out which of these websites are specifically forbid automated data extraction. Here’s is my result:

1. COURTS
   Allow Filter by RAM Size: NO
   Filter Laptops by: Price, Brand, Customer Rating, Laptop Type, Screen Size, Processor and Color
   Allow Data Scraping: NO

   You may not extract and/or re-use the Brand Names or parts of the content of COURTS Online without our express written consent. In particular, you may not use any data mining, robots, or similar data gathering and extraction tools to extract (whether once or many times) for re-use of any substantial parts of the content of COURTS Online, without our express written consent. You may also not create and/or publish your own database that features substantial parts of COURTS Online (e.g. prices and product listings) without our express written consent.

   COURTS CONDITIONS OF USE AND BUYER AGREEMENT

2. HARVEY NORMAN
   Allow Filter by RAM Size: NO
   Filter Laptops by: Price, Brand, Processor and Type
   Allow Data Scraping: No explicit prohibition regarding automated data extraction (Term Of Use)

3. LAZADA
   Allow Filter by RAM Size: NO
   Filter Laptops by: Price, Brand, Customer Rating, Laptop Type, Screen Size, Service and Location
   Allow Data Scraping: No explicit prohibition regarding automated data extraction (T&C)

4. BEST DENKI
   Allow Filter by RAM Size: NO
   Filter Laptops by: Price, and Brand
   Allow Data Scraping: No explicit prohibition regarding automated data extraction (T&C)

Now that I’ve got the legal foundation covered, my next step is to actually build the web-scraper code. See you again soon!

Recently I found this data laying around in my Projects folder. I believe I scrapped Wikipedia to have this data. Clearly there is no much use of this data at the moment. Maybe just to showcase how JSON Table Editor handles unicode characters.

Anyway, enjoy this historical data.

To ensure you’re on the right server, run this command:

cat /etc/os-release

You should see something like this:

Then download the rpm file (the latest version doesn’t have the rpm file)

wget https://www.nano-editor.org/dist/v2.5/RPMS/nano-2.5.3-1.x86_64.rpm

Then install the package

rpm -i nano-2.5.3-1.x86_64.rpm

That’s all 😀