The European Consumer Centre’s Network.
Link to The Competition and Consumer Authority.

Front page | | Contents | | Bottom | | <<Previous | | Next >> |

3. Research methodology

3.1. Organisation and planning

The working group had its first meeting in October 2010 at the office of ECC Sweden in Karlstad. At the meeting the criteria for the websites and the product categories were determined as well as the layout for the overall project process and the distribution of tasks among the working group members.

3.2. Product categories

In order to determine the most relevant product categories, we consulted the Consumer Market Scoreboard[19] to find the most popular categories. The following 10 relevant product categories were identified:

  1. Clothing
  2. Sporting goods
  3. Household goods (e.g. blender, lamp etc.)
  4. Books
  5. Music CDs
  6. DVD films
  7. Video or computer games
  8. Computer software (non-downloadable)
  9. Electronic equipment (e.g. memory cards, cameras, mp3 players, consoles, external hard discs etc.)
  10. Products for personal care (e.g. lotions, shampoos, perfumes etc.)

3.3. Selection of websites

All 29 ECCs were asked to submit a list of 40 web traders from their own countries. The premise was that it would be faster and easier for us to find appropriate web traders in our respective countries. For the purpose of selecting the web traders, the working group drafted a guide, including instructions on a minimum set of criteria. The ECCs were instructed that the web traders had to be willing to sell cross-border. The web traders had to accept credit or debit cards as a method of payment and the website had to operate in at least two languages. When searching for relevant web traders, the ECCs were also asked to make sure that some of the web traders were members of a Trustmark scheme. We wanted to check whether this would have any effect on the trader’s compliance with EU law.

In 2002-2003 it had been difficult to find enough web traders offering cross-border sales. Even with a sample size that was larger than that of the 2003 report, it was our belief that finding enough relevant web traders for this project would not be a problem this time as we assumed that a larger number of web traders offer cross-border sales compared to 2002-2003. This assumption was not changed despite the fact that, according to the Consumer Conditions Scoreboard, the share of web traders advertising and selling across borders has decreased over the past few years[20].

We were quite surprised to find that many web traders still prefer only to sell products to their domestic market and for some ECCs it was simply impossible to submit a list of more than just a few web traders that fulfilled the criteria described above. This caused a great deal of problems for the project participants. Firstly, because the search for web traders in itself was more time consuming than originally presumed and, secondly, because many websites had to be substituted during the shopping exercise. These problems will be further analysed in Chapter 7 (“Website problems prior to purchase”) of this report.

3.4. Distribution of websites

After receiving all the lists of suggested web traders, the working group distributed a list of 20 websites to each participant. The distribution was based on different criteria. The working group decided that it would make sense that more orders be placed in countries with the highest number of submitted web traders. The working group also took into account the complaints submitted to the ECC-Net[21] in order to reflect reality as much as possible. The statistics presented the top 12 ECC countries to which each project participant country sent the most complaints during the first three quarters of 2010. Therefore, the working group decided that the Mystery Shoppers had to make more purchases in those countries where the ECC of the Mystery Shopper had sent the most complaints. Further, we tried to take into account a principle of neighbouring countries, with the reasoning that consumers are more likely to shop across borders with their direct neighbours due to the shorter distance and a lower risk of language barriers.

3.5. Shopping exercise

It was decided that each Mystery Shopper would make 20 purchases. This would make the total number of websites to shop from 340. The Mystery Shoppers were given a comprehensive shopping guide explaining to them how to carry out their 20 purchases. The purchases were made by individuals, using their private credit cards and the products were delivered to their home addresses. The Mystery Shoppers were recommended to make each purchase within a price range of EUR 50-150, where possible, as it was the assessment of the working group that this would be a realistic price range for making cross-border purchases and that it would also serve to minimise the risk of losing very large amounts of money.

3.6. Questionnaire

The project participants were instructed to collect all the relevant data obtained during the shopping exercise in a questionnaire. The questions were grouped into four different categories in what was deemed to be the most logical chronological order:

  1. Information pertaining to the purchase
  2. Information pertaining to delivery
  3. Information pertaining to the return and refund
  4. CPC and other issues

In order to obtain countable results, we pre-selected a range of replies to the questions wherever possible[22]. The questionnaire also contained an additional field for individual remarks at the end of each subsection. The questionnaire is submitted as Appendix 1 to this report.

3.7. Limitations

It was decided not to include the purchase of services in the project as this would give rise to too many concerns about the right of withdrawal as well as add further limits to the test of returns and reimbursement. It was also decided that orders which had been placed but where the product had not been delivered and the purchase price had not been withdrawn would not count as a purchase in the questionnaires. Instead, these orders will be dealt with separately in Chapter 7 (“Website problems prior to purchase”). Therefore the project ended up with fewer purchases than originally planned[23].

In the shopping guide, the project participants were instructed to carry out the information checks and to keep documentation (print screens) of certain steps of the purchase process. Mystery Shoppers were also instructed to document any unusual occurrences during the process. Bearing in mind that we could end up encountering issues that would be of interest to the CPC-Net, we wanted to be able to provide the necessary documentation.

[19] Cf. “Consumer Markets Scoreboard: Consumers at Home in the Internal Market”, 4th edition, October 2010, p. 11-13, and “Consumer Markets Scoreboard: Making Markets Work for Consumers”, 3rd edition, March 2010, p. 22.

[20] According to p. 6 of the Consumer Conditions Scoreboard (see footnote 1), the share of web traders advertising and selling cross borders actually decreased from 25% in 2009 to 22% in 2010.

[21] For this purpose, statistics were retrieved from the IT TOOL. The data was submitted to the working group by DG SANCO.

[22] Note: For the sake of clarity in the presentation of the results gathered from the questionnaire, it was decided to round up or down the calculated percentages.

[23]  Note: Two Mystery Shoppers experienced a great deal of problems and due to different circumstances they ended up only being able to conduct 10 purchases each. Due to these unexpected circumstances combined with the decision to exclude purchases where no product was delivered and no money was withdrawn, the total number of purchases completed was further reduced, making the final figure upon which the conclusions in this report are based 305 purchases.

Top |

This page forms part of the publication "ONLINE CROSS-BORDER MYSTERY SHOPPING – STATE OF THE e-UNION" as chapter 2 of 8.
– STATE OF THE e-UNION" as the preface.
– STATE OF THE e-UNION" as appendix -6.
Version no. 1.0, 2011-08-31
© The Competition and Consumer Authority