VPN Series Appendices

Appendix A:
Correlation between VPN review site ratings and commissions

The details of my own research (in late 2019) regarding the correlation of the ratings of VPN review sites and the commissions those sites receive from VPN companies is presented. This includes a table of the raw ranking data that I collected and also a list of VPNs and the commission rates that they offer to VPN review sites. Also, I describe my assessment of that data.

Rankings of the “Best” VPNs by review websites

I performed an internet search for “best VPN” and loaded the top “hits” of VPN review websites that resulted. I examined 12 different VPN review websites and recorded the “Top 5” rated VPNs that were listed on each site. There were a total of 19 different VPNs that appeared on one or more of the twelve “Top 5” lists.

Next, I created a master list of the VPN rankings, noting the total number of times a VPN appeared on a “Top 5” list of all of the VPN review sites and also the average rank it received for all the sites on which it was listed. I tallied 5 points for the VPN if it was listed in first place on a “Top 5” list, 4 points if it was listed in second place, etc.

Thus, using this ranking method, the very best result for a VPN would be for it to be listed on all 12 of the 12 different review sites and to be in first place on every “Top 5” list (yielding a 5.0 point average rank).

Following is the list of the VPN rankings that I compiled:

Column content explanation:

VPN Name: the name of the VPN
1-12: the rank (1-5) given to the VPN by that numbered review site (see below for list of review sites that correspond to the numbers 1-12)
Total: the sum of my ranking points tally from all of the review sites
Occurs: number of times (occurrences) the VPN was listed as a “Top 5” VPN
Ave.: the average ranking point value for the VPN per site that listed the VPN (if more than 1 occurrence)

VPN Name	1	2	3	4	5	6	7	8	9	10	11	12	Total	Occurs.	Ave.
Astrill VPN										3			3	1
CyberGhost	4	3		1		3	3	3	3		3	2	25	9	2.8
ExpressVPN	5	5	5	5	5	4	5	5	5	5	5	4	58	12	4.8
HideMyAss!	3					1							4	2	2.0
Hotspot Shield				2				1					3	2	1.5
IPVanish		2		4	2		1				2		11	5	2.2
Ivacy										1			1	1
Mullvad					1							1	2	2	1.0
NordVPN	2	4	3	3	4	5	4	2	4	4	4	5	44	12	3.7
Perfect Privacy			4		3								7	2	3.5
P.I.A.									2				2	1
PrivateVPN		1				2					1		4	3	1.3
SaferVPN	1												1	1
Surfshark								4					4	1
TorGuard									1	2			3	2	1.5
TunnelBear												3	3	1
VPN.ac			1										1	1
VPNArea			2										2	1
VyperVPN							2						2	1

List of the 12 VPN review sites:

Column #1 = securethoughts.com/best-vpn/
Column #2 = www.comparitech.com/blog/vpn-privacy/best-vpn-for-mac/
Column #3 = restoreprivacy.com/best-vpn/
Column #4 = www.techradar.com/vpn/best-vpn
Column #5 = thebestvpn.com
Column #6 = www.vpnmentor.com/bestvpns/overall/
Column #7 = www.bestvpnrating.com/review
Column #8 = top5-vpn.com
Column #9 = www.cloudwards.net/vpn-reviews/
Column #10 = vpnpro.com/best-vpn-services/
Column #11 = www.top10vpn.com/best-vpn/
Column #12 = www.drivereasy.com/knowledge/the-best-vpn-services-the-newest-review/#vpns

VPN affiliate program commission rates - 19 examples

Next, I visited the websites of multiple VPN services, looking to see if they paid for referrals of new customers. I found this to be a very common practice.

Following is a list of the commissions that VPN service companies pay to VPN review sites if a person referred by the VPN review site becomes a paying customer of the VPN service.

This contractual arrangement between VPNs and VPN review websites is known as an “affiliate” program and amounts to a “kickback” to VPN review sites.

Commissions are typically a percentage of the yearly service fee that the customer pays to the VPN.

Perfect Privacy VPN: 1 year @ 30% = $41.00

IPVanish: 1 year @ 50% = $39.00 (“up to 100% of all new sales generated.”)

SaferVPN: 1 year default rate = $37.00

Express VPN: 1 year default rate = $36.00 (“The more sales you drive, the more we pay.”)

VirtualShield: 1 year = $36.00

Nord VPN: 1 year @ 40% = $33.60

HideMyAss VPN: 1 year @ 40% = $33.55

CyberGhost: 1 year up to $31.50 (??)

Surfshark: 1 year @ 40% = $28.75

Private Internet Access (PIA): 1 year @ 33% = $23.72

Trust.Zone VPN: 1 year @ 50% = $20.00

TorGuard VPN: 1 year @ 30% = $18.00

CyberSilent VPN: 1 year @ 25% = $16.93

Ivacy: 1 year @ 40% = $16.00

PureVPN: 1 year @ 40% = $16.00

Cactus VPN: 1 year @25% = $13.75

Boleh VPN: 1 year @ 15% = $12.00

Liberty VPN: 1 year = $10.00 (“Earn up to $9.99 commission per sale.”)

Saturn VPN: 1 year @ 30% = $4.80

As can be seen from that list, there is a very wide range of commission payments by VPNs to VPN review websites. The highest commission is 850% of the lowest payment.

VPNs with no affiliate program

There were only three VPN services for which I could find no evidence of an affiliate program in late 2019 that (as of early 2023) continue to not support an affiliate program. Those VPNs are:

CryptoStorm

IVPN

Mullvad

Analysis of VPN ratings with respect to commission rates

Let’s analyze this data from different perspectives.

Correlation of inclusion on “Top 5” lists with commission rate

From my own chart “Rankings of the ‘Best VPNs’ by review websites” (19 VPNs made it to the chart), the VPNs with the most occurrences on the “Top 5” lists of the VPN review sites were:

VPN Name	occurrences on the "Top 5" lists
ExpressVPN	12 occurences
NordVPN	12 occurences
CyberGhost	9 occurences
IPVanish	5 occurences

(No other VPN had more than three occurrences.)

Checking the VPN affiliate program commission rates, I found that all of the above four VPNs were amongst the 8 highest paying of 22 (affiliate + non-affiliate) VPNs.

Of the 10 lowest paying (affiliate + non-affiliate) VPNs, only two VPNs made it to the chart:

• Ivacy had one occurrence on the 12 “Top 5” lists
• Mullvad had two occurrences on the 12 “Top 5” lists

This implies that the “price of admission” to a “Top 5” VPN list is a high commission payment to VPN review sites.

Correlation of rank on "Top 5" lists with commission rate

The two VPN services that were included in the list of each and every one of the 12 VPN review websites that I surveyed were ExpressVPN and NordVPN. CyberGhost was the third most commonly listed VPN, occurring in 9 of the 12 lists.

• ExpressVPN was ranked higher than NordVPN on 10 of the 12 “Top 5” lists
• ExpressVPN pays a higher commission to VPN review websites than NordVPN pays

• CyberGhost was ranked lower than both ExpressVPN and NordVPN on all but one of the lists on which it appeared.
• CyberGhost pays the lowest commission rate of these three VPNs

Furthermore, the 5 VPNs with the highest average ranking, as assigned by the VPN review websites, were:

• ExpressVPN
• NordVPN
• PerfectPrivacy
• CyberGhost
• IPVanish

The median commission for all of the other VPNs was only $18.00, i.e. half that of the highly-ranked VPNs.

This implies that payment of a high commission to VPN review sites results in a high ranking on “Top 5” VPN lists.

Appendix B:
Trackers and cookies on VPN websites

This section details my own research on the use of web trackers and cookies found to be present on the websites of VPN providers. Methodology, raw data and analysis are presented.

Testing procedure

To test for trackers and cookies on VPN websites, I visited the main web page of the 50 VPN services listed in the “VPN Comparison Table” that I found via the reddit.com r/VPN community.

This table is a recently compiled and seemingly unbiased attempt to assess VPNs for desirable characteristics, resulting in a ranking of the VPNs. See the post about this table on the reddit.com r/VPN community.

My methodology was to visit the main page of each of the 50 VPN services listed in that “VPN Comparison Table.” I loaded each page with two different browsers: Firefox and Safari. Running macOS Ventura v. 13.0.1, I used Firefox v. 106.0.2 with the settings of Firefox set to “Standard Tracking Protection” and with the EFF Privacy Badger extension (version 2022.9.27) enabled. When using Apple’s Safari v. 16.1, “Prevent cross-site tracking” was enabled.

In Safari, I simply noted the number of trackers that Safari reported that it had “prevented from profiling you.”

In Firefox, I tabulated the tracking report from the EFF Privacy Badger extension. This extension “stops advertisers and other third-party trackers from secretly tracking where you go and what pages you look at on the web.”

Privacy Badger assigns a threat level to the trackers that it finds on websites. “Red” trackers are considered to have the highest threat level and to have no functionality other than tracking your web browsing. “Yellow” trackers are considered to likely be adding some functionality to the website but are also tracking your web browsing.

Thus, websites with “red” trackers are purposely designed to compromise your privacy and websites. The grading system that I developed (see below) strongly downgrades VPNs that were found to have “red” trackers.

Also in Firefox, with respect to cookies, I noted the number of cookies that were saved by the browser upon visiting the VPN website. Some of these are “session” cookies and are functional and not necessarily problematic.

Furthermore, in Firefox, I recorded the number of persistent cookies, as reported in “Cookies and Site Data” in the “Privacy and Security” section of Firefox settings. “Persistent cookies” refers to the number of cookies attributed to the VPN that were still present after closing all browser windows, quitting Firefox and then restarting Firefox. These persistent cookies threaten your privacy.

I found these results of my tracker and cookie testing to be highly disconcerting: there’s a lot of red in that chart! Indeed, of the 50 VPN websites that I tested, 30 (60%) of them exhibited severely problematic tracking behavior!

A mere three VPN websites (Cryptostorm, Mullvad and VPN.ac) respected the privacy of visitors to their websites, employing no tracking technology. (Another seven VPN websites did not use trackers but did use one to three persistent cookies.)

The abysmal privacy invasion by the majority of VPN websites is unwarranted and deplorable.

Appendix C:
That One Privacy Site

The voluminous data about VPNs collected and presented by the anonymous “That One Privacy Guy” was an early source of unbiased information about VPNs. Unfortunately, the website closed in late 2020. Although gradually becoming dated, some of the data remains relevant and useful.

Update about the changes to this previously reliable source

When I first wrote my series of articles about finding a trustworthy VPN in early 2019, there were very few principled sources of information about VPN services that I had found. These reliable sources were referred to repeatedly in my series of articles. Unfortunately, the VPN commentary and comprehensive database of one of those trustworthy sources has been sold to a VPN review website that holds a distinct conflict of interest.

The information which I had previously found to be so useful was originally found on Reddit.com and on the “That One Privacy Site” website. Compiled by an anonymous individual known as “That One Privacy Guy,” this source of information about VPN services included a useful guide to choosing a VPN, as well as comprehensive VPN comparison charts.

The VPN information on the “That One Privacy Site” website was generally considered to be devoid of the biases rampant on other privacy-oriented websites.

However, in late 2020, “That One Privacy Guy” closed his own website and sold out to a VPN review website that is now known to be owned by a holding company which owns at least four VPN services. This is a blatant and critical conflict of interest. Hence, any reviews of and recommendations for VPN services on that new VPN review website must be considered to be biased and untrustworthy.

Since the website changeover occurred, the very useful “Choosing a VPN” guide, written by That One Privacy Guy, has been deleted from the new hosting website. Consequently, I have modified the links in my articles that refer to the (still relevant) original information in that guide to point to the archived copy of the “Choosing a VPN” guide that can still be found on The WayBack Machine Internet Archive.

The comprehensive databases of information about VPNs (which are derivatives of the “Simple” and the “Detailed” “VPN Comparison Chart” that were originally compiled by That One Privacy Guy) are available on the new website. However, I have considerable doubt about their veracity.

Fortunately, the original “Simple VPN Comparison Chart” and the “Detailed VPN Comparison Chart” were released under a Creative Commons license. The license allows anyone who follows its terms to “copy and redistribute the material in any medium or format.”

Under these license terms, I am making the “Simple VPN Comparison Chart” and the “Detailed VPN Comparison Chart” available for viewing and/or downloading on my website. The source of these charts from the “That One Privacy Site” by “That One Privacy Guy” is archived at That One Privacy Site archive.

The links in my articles that refer to the “Simple VPN Comparison Chart” and the “Detailed VPN Comparison Chart” have been updated and will no longer bring you to the charts hosted on an external web archive site or on the new review website. Rather, the links will point to copies of the VPN Comparison Charts (that I had saved in the past) and am making available under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license on my yourmacdoc.com website.

The VPN Comparison Chart versions that I am using date to September 2019 and are the last versions of the charts that I consider to be trustworthy. These earlier versions of the charts, though somewhat dated, are considered to be unbiased.

Although the Simple and Detailed VPN Comparison Chart data was compiled in 2019, the charts remain relevant today. In particular, the “grades” assigned to the nine broad categories for each VPN in the “Simple” chart are likely to be stable over the long term.

E.g., it is fairly likely that a VPN that earned a good grade in the Business Ethics category in 2019 will still be ethical today.

Even parameters that change can still remain relevant. E.g., almost all VPN services are steadily increasing the number of servers to which their customers can establish an encrypted connection. Hence, the number of servers listed in the chart for each VPN is no longer accurate. However, the relative differences in the number of servers when comparing VPNs will likely remain similar. Thus, a VPN that earned a good grade for Availability in the past will probably continue to have more servers available today when compared to a VPN that earned a poor Availability grade in the past.

Therefore, the Simple and Detailed VPN Comparison Charts of “That One Privacy Guy” are still useful today, even though some of the data in the charts is becoming outdated.

Appendix D:
Consumer Reports security and privacy testing of VPNs

In late 2021, a thorough study of VPN services was published by Consumer Reports, an independent, nonprofit organization that is a well-respected non-biased source of reviews of a wide range of products. Two articles on the Consumer Reports website and a more formal “white paper” were used to present information about the VPN testing that was performed.

This Consumer Reports study meets all of my criteria for being a valuable resource of information about VPNs. The study is comprehensive, rigorous, objective (non-biased) and was conducted by a non-profit organization that is well-known for its reliable and trustworthy investigations of products and services. Furthermore, the study, being of recent origin, is up-to-date with respect to the VPN services industry.

The study initially surveyed over 200 VPNs. That list was pared down to 51 VPNs by only including VPN services with a large market share or VPNs that had evident markers of quality, such as public ownership, open source code, public third-party security audits, support for modern protocols, and accurate ad copy.

The group of 51 VPNs then underwent further screening, which narrowed the final list to 16 VPNs. A comparative privacy and security evaluation was then conducted on those 16 VPNs on the final list.

The 16 VPNs that survived the winnowing process and underwent final comprehensive testing were:

• Betternet
• CyberGhost
• ExpressVPN
• F-Secure Freedome VPN
• Hotspot Shield
• IPVanish
• IVPN
• Kaspersky VPN
• Mozilla VPN
• Mullvad
• NordVPN
• Private Internet Access (PIA)
• Private Tunnel
• ProtonVPN
• Surfshark
• TunnelBear

It is important to note a potential shortcoming of the selection process that produced that list of 16 VPNs: VPNs secured a slot on that list based on having a “larger market share” or based on having “markers of quality.” Hence, there are VPNs on that list of 16 that were included solely for their prominent “market share.”

Unfortunately, the “VPN White Paper” report does not specify how each VPN qualified for that final list of 16 VPNs. However, one would hope that the final process of extensive testing that was performed (as noted below) would result in any poor quality VPNs with high market share being revealed by their mediocre test performance.

Each of the 16 VPN services on the final list were subjected to a privacy and security evaluation that used the criteria of the Digital Standard, a “framework to evaluate how technologies respect consumers’ interests and needs.” “The Digital Standard is a living, open source consensus used by independent testing organizations, researchers and product teams.”

The extensive comparative testing of each VPN service included security testing and data privacy testing:

 Security Evaluation

• Build Quality: Best Build Practices
• Effectively Implemented Safety Features
• Open and Reproducible Software
• Authentication
• Encryption
• Known Exploit Resistance
• Security Oversight
• Security Over Time
• Vulnerability Disclosure Program

 Data Privacy Evaluation

• Access and Control: Data Control
• Data Use and Sharing: Data Sharing
• Data Use and Sharing: Data Use
• Data Retention and Deletion
• Overreach/Collecting Too Much Data: Data Benefits
• Overreach/Collecting Too Much Data: Data Collection
• Overreach/Collecting Too Much Data: Minimal Data Collection
• Overreach/Collecting Too Much Data: Privacy by Default
• Governance: Privacy Policies and Terms of Service
• Governance: Privacy Policy and Terms of Service Update Notification

The results of nearly all of these tests were presented in bar graph format to compare each VPN with the others.

Many other technical and non-technical issues were addressed in the report, including:

• Local Logging
• Dark Patterns
• Human Rights and Corporate Social Responsibility
• Logging
• Inaccurate Presentation of Products and Technology
• Sweeping or Overly Broad Anonymity and Privacy Claims
• Claims of Military-Grade Encryption
• Most Accurate Presentation of VPNs, Their Uses, and Their Underlying Technology
• Ownership
• Transparency Reports
• Complaints
• VPN-Owned VPN Review Sites
• Response to Breaches
• Best Practices

As you can see from the above lists of issues addressed by this Consumer Reports VPN White Paper investigation exposition, this study of VPNs is extremely comprehensive.

In my opinion, the study is second-to-none, and, as of this writing in early 2023, it is the gold standard for unbiased, comprehensive and trustworthy information comparing VPN services.

The end of the report states:

Recommendations for Users

Of the 16 VPNs we analyzed, Mullvad, PIA, IVPN, and Mozilla VPN (which runs on Mullvad’s servers)—in that order—were among the highest ranked in both privacy and security. However, PIA has never had a public third-party security audit.

Additionally, in our opinion, only IVPN, Mozilla VPN, and Mullvad—along with one other VPN (TunnelBear)—accurately represent their services and technology without any broad, sweeping, or potentially misleading statements.

The Consumer Reports Security and Privacy of VPNs Testing Links

I strongly recommend that you make the effort to thoroughly study the “VPN White Paper” report that is linked below. It offers valuable lessons on how VPN services should be evaluated and critiqued.

• Security and Privacy of VPNs - Consumer Reports VPN White Paper (This is the full report of the VPN comparative testing in .pdf format.)
  
  
• VPN Testing Reveals Poor Privacy and Security Practices, Hyperbolic Claims (This is the article on the Consumer Reports website that, in general, discussed the VPN testing and presented the results.)
  
  
• Mullvad, IVPN, and Mozilla VPN Top Consumer Reports’ VPN Testings (This is the article on the Consumer Reports website that focussed on the results of the VPN testing.)

Appendix E:
The Wirecutter review of VPN services

Wirecutter (acquired by The New York Times in 2016) has been publishing product reviews since 2011. Wirecutter reviews consist of detailed guides of a wide range of consumer products. The articles typically focus on just two or three of the products that the review process has found to be the best of the category.

Wirecutter participates in affiliate programs, earning commissions on many of the product it recommends. However, as noted in the Wirecutter Wikepedia article, “to prevent bias, the staff who write its reviews are not informed about what commissions, if any, the site receives for different products.”

Having used Wirecutter as a source for product reviews over the past several years, I have found its reviews to indeed be unbiased and authoritative. In contrast to my opinion about almost all VPN review websites, I consider the Wirecutter VPN review articles to be trustworthy.

The breadth and depth of the Wirecutter VPN reviews are outstanding. Note the sections in their January 2023 update to their VPN review article:

• Why you should trust us
• Who this is for
• What you should do before considering a VPN
• Geoshifting
• Trusting a VPN
• Limitations of VPNs
• How we picked
• How we tested
• Our pick: Mullvad
• Also great: TunnelBear
• What about HTTPS?
• What about Tor?
• What about creating your own VPN?
• What to look forward to
• The competition
• Frequently asked questions
• Sources

Thus, the Wirecutter VPN reviews serve to validate the reviewer’s status, educate the reader about VPNs, explain the review methodology, discuss the features of the best VPNs, note alternatives to VPN services and document their sources.

The Wirecutter VPN review articles are updated on a regular basis. Since 2018, updates have been issued once or twice every year.

The January 2023 Wirecutter VPN review process started with 57 VPNs. These were screened with a list of multiple strict criteria that were considered important for a trustworthy VPN service.

Wirecutter's VPN screening criteria:

• Trust and transparency
• Privacy and terms-of-service policies
• Trial or refund policy
• Server network
• Security and technology
• Kill switch
• Platforms
• Number of connections
• Support
• Extra features

After the screening process, only four VPN services met the criteria for further review: Mullvad, Surfshark, TunnelBear and IVPN.

Having already met the challenge of the screening process, the four VPN services underwent further testing.

Wirecutter's testing of VPNs on the final list:

• Speed testing - using eight different server locations per service
• Efficacy checks - testing for IP address, DNS and WebRTC leaks and website blocking
• Desktop and mobile apps - evaluation of the user interface of the VPN software and the payment process
• Customer support - a live test of contacting customer support as well as an assessment of online help/support

Wirecutter's recommendation for the “Best VPN Service”

Mullvad was Wirecutter’s “pick” for best VPN service. “After sorting through dozens of VPNs and reviewing four security audits, we think the best option for most people is Mullvad, an open-source VPN that’s not only trustworthy and transparent but also fast and reliable. The company excelled in signals of transparency and trust, and in our testing the service was easy to use and delivered some of the fastest speeds of any VPN we tried.”

Wirecutter’s “also great” (runner-up) VPN choice, TunnelBear, was noted to be “transparent with its practices, implementing annual security audits that also offer a detailed view into what steps it takes to protect its OpenVPN-based infrastructure.” Of note, TunnelBear does not support the state-of-the-art WireGuard VPN protocol.

Like the Consumer Reports Security and Privacy of VPNs study, the Wirecutter review meets all of my criteria for being a valuable resource of information about VPNs. (The Consumer Reports “white paper” report seems to be written for a somewhat more technically-oriented audience than the Wirecutter VPN review.)

The Wirecutter review serves as a single all-purpose source for someone looking to choose a VPN service. Consumer Reports relies on its supplemental articles to the VPN “white paper” to provide introductory and background VPN information to its readers.

Wirecutter VPN Review Links:

The Best VPN Service (This comprehensive VPN review includes a very thorough introduction to VPN services. This article is an excellent educational resource about VPNs.)

What Is a VPN and What Can (and Can’t) It Do? (A primer for VPN services that provides basic, reliable background information.)

Summary

These appendices supply supplemental information about the sources that were used to produce the “Choosing a Trustworthy VPN” series of articles.