Spiekermann, Sarah: Online Information Search with Electronic Agents: Drivers, Impediments, and Privacy Issues

35

Chapter 3. Empirical Work

3.1 Overview

In order to investigate consumers‘ targeted online information search behavior for high-involvement goods an experiment has been carried out in winter 2000 at Humboldt University Berlin. The experiment was designed as an ordinary shopping trip to an experimental online store where participants could shop either for a winter jacket or for a compact camera. 206 subjects were observed through registration of log-file data in their search behavior. Besides manually controlled information zones, the shopping trip was supported by a selected options-based anthropomorphic advisor agent.

The total sample of 206 was split into the two buyer groups. In addition, two different types of privacy regulations (type 1 and type 2) were employed in the store offering shoppers more or less comfort with data handling policies. As a result of these different buying conditions (product and privacy regulation), four treatments summarized in table 1 can be discerned for the empirical research.<16>


36

Tabelle 1: Overview of Experimental Treatments:

Treatment No.

Product sought

Privacy Statement confronted

Number of observations made for treatment

1

Camera

Type 1

60

2

Camera

Type 2

92

3

Jacket

Type 1

28

4

Jacket

Type 2

26

The first goal of this analysis(conducted on the basis of experimental data) was to gain insights into the drivers of, and impediments faced by online information search. In particular, when combined with electronic advisor agents. Would interaction levels with an agent be explicable by the same factors as manual driven search? What is the relative importance of agent-based versus manually driven search? In order to investigate these questions, a structural equation model of online information search has been developed and observations from camera-treatments 1 and 2 were used to test it (analysis 1, table 2). The dependant variables in this model were agent-based search and user driven search.

The main reason why only data from camera shoppers have been included in this analysis is that information search behavior was shown to differ between product groups [Kotler, 1994]. And even though compact cameras and winter jackets were similar in value for the experiment, they represent two very distinct types of goods: While jackets entail relatively strong characteristics of an experience good, meaning that consumers like to judge on product quality through feeling and touching different models, cameras display stronger search-good characteristics, meaning that consumption benefits can be predicted more reliably prior to purchase on the basis of factual product information (for the distinction of goods see [Nelson, 1970]). As a result, cameras represent a product class for which the Internet offers relatively strong information advantages. It therefore seemed advisable to not intermingle behavioral findings for this product class with the observations made for jackets (treatments 3 and 4). Restrictions in the size of the dataset for jackets (only 54


37

observations) prevented a separate calculation and test of the equation model for this product category.

In order to still take account of the impact of product nature on interaction, a second round of analysis was conducted including data from all treatments, camera and jacket shoppers (analysis 2). Leaving behind the more macro-level type of behavioral analysis inherent in structural equation modeling, the information search process for cameras and jackets was analyzed in more detail. For this purpose, the dependent information search constructs, agent-based search and manually controlled search (measured in the equation model on the basis of time and page requests) were pulled apart to smaller pieces of search activity (such as the number of photo enlargements made by a subject). Then the impact of product nature on this micro-level type of search activity was analyzed.

Besides the investigation of ’high level‘ relationships relevant in online information search, one potential impediment for interaction with agents has been investigated in more detail. This was the privacy concern of online consumers. Privacy concerns turned out to be the most important impediment for agent interaction in the structural equation model. As was mentioned above, they have also been proclaimed as an important factor for agent acceptance [Shearin and Maes, 2000; Norman, 1994]. Investigating privacy issues in agent interactions, the first step was to capture the phenomenon of privacy concern in a regression model. An index was developed that aims to capture personal consumer information cost (PCIC) (analysis 3). Based on this index (as well as other variables), expressed privacy preferences were then compared with actual interaction behavior (analysis 4) during the shopping session.

Table 2 gives an overview of the different analyses made and the number of observations used to make them. All of them will be reported on in more detail in the following chapters of this thesis. The following sections of this chapter will give a more detailed insight into how the experiment was conducted.


38

Tabelle 2: Overview of Empirical Analysis Made:

Analyses

Results reported in

Treatments
included

N° of Observations used to make analysis

1. Structural Equation Model of Online Information Search

Chapter 4

1 and 2

152

2. Product Nature and Online Information Search

Chapter 5

1, 2, 3, 4

206

3. Regression Model of Privacy Concerns

Chapter 6

Separate study

39

4. Privacy Concerns and Actual Behavior

Chapter 6

1, 2, 3, 4

206

3.2 Incentive Scheme and Briefing

The experiment was advertised at Humboldt University Berlin. Its goal was described as a test of user interaction with a highly innovative and high performance product search engine developed for online shopping systems at the Institute of Information Systems at Humboldt University. Students were told that the system would be tested on the basis of a ’real-world‘ shopping trip for cameras and winter jackets. If people were interested to buy a camera or winter jacket they were asked to sign up for one of the shopping sessions taking place from a computer laboratory at a pre-arranged time.

A major challenge in winning participants for the experiments was to find people with a true interest in one of the two products. As was discussed in section 1.1., the use of money or class credits are a questionable incentive structure when interaction with and reliance on agents is being tested. Participants‘ interest in the product was assured by making them pay for products out of their own pocket if they wished to buy something. The main incentive to purchase (and participate) was a 60% discount on all products on offer in the store. Unfortunately, project finances were constrained


39

could not offer this discount to all buyers. The incentive structure therefore had to be refined such that a lottery, after shopping, decided (on the basis of one out of ten participants in a lab session) who would have the right to take a product for the 60% off. The remaining participants had the choice to still buy the product for the full price displayed and received a small financial compensation of 20 DM (approximately $ 10) to reward them for their efforts. If someone decided to not buy anything in the online store, but won the lottery, he or she would leave empty. With this incentive scheme in place the desire to purchase was realistic. .Due to the high value nature of cameras and jackets, buyers had to expect, with a relatively high chance of 1:10, to incur a minimum expenditure of 80 DM (approximately $ 40). Participants were made even more aware that they had to pay for purchases by have to sign obligation to pay forms prior to shopping. People who may have come to only cash in the compensation without buying anything were discouraged by the fact that with the same 1:10 chance they would have to bank on leaving empty. In addition to the discount, participants were also promised a personal feedback on their interaction behavior.

Winning experimental participants by offering a product discount and feedback on behavior led to a random self-selection process for all treatments. 92,7% of total participants were students from different university faculties while the remaining 7,3% held different jobs. 55,8% were male and 44,2% female. 98,5 % of the participants indicated to have experience with the Internet and 91,7% of them would even regularly use it (for SPSS output file see Appendix D, table D1). With these user traits in the data the advantage is that a relatively well-educated cross-section of the population with considerable online experience has been observed. Online behavior cannot be attributed to the ’naivety‘ of subjects in interacting with the online system. Also, a relatively prominent target segment for today‘s electronic commerce environments has been investigated: 56,3% of the participants indicated that they had already bought something online. Given these demographics and the characteristics of the participants, a disadvantage of the experiments is that the sample is not representative of the German population or consumers in general. In addition, only those people that are relatively open to use direct marketing channels such as the Internet, (and are thus ready to handle the risk of not being able to touch and feel the product before buying) may have registered for the experiments.


40

When the term ’consumer‘ or ’Internet user‘ is employed in the following sections to comment on observed behavior, then this generalization is made only to facilitate the description of relationship and reading of the thesis. The ’student‘ as a particular type of consumer observed and referred to should be kept in mind by the reader.

3.3 Materials and Apparatus

The central material for the experiment was an online store with two different versions, one offering compact cameras and the other one offering winter jackets (for screenshots see Appendix B1). In addition to this online part of the experiment, a battery of questions was answered by participants before and after the shopping session which was identical for jacket and camera shoppers (Appendix A5). The shop‘s functioning was tested twice before the experiment took place. The first tested the enhanced store design and layout, and the second the performance of the recommendation algorithm.

3.3.1 Navigation Opportunities in the Experimental Online Store

The experimental online store was programmed explicitly for the experiment, using Meta-HTML and Java. In order to encourage product search, the shop had a vast range of models on sale including 48 compact camera models and 100 winter jackets (50 models displayed to women and 50 to men). The reason why there were so many different models on offer for each product was that the agent was intended to be highly responsive to users‘ expressed product preferences, making the benefits of interaction visible for participants. At the same time, participants were let to feel slightly overwhelmed by the volume of alternatives giving them an incentive to invest themselves into the search process.

The interactive opportunities participants encountered in the store were similar to those in website like ActiveBuyersGuide.com or PersonalLogic.com. The online store‘s starting page had been loaded into the Web browser by the experimenters when the participants arrived at the lab. It displayed either a camera or a jacket storefront depending on the product the subject had registered for. In the store


41

navigation was organized in three phases: When participants entered the online store they had a space for orientation (phase 1) where they had the possibility to view all products on offer, one by one, from a list. From there, users proceeded to the agent domain where an anthropomorphic 3-D advisor agent (“Luci“) enrolled the user in a communication or interaction phase (phase 2). The interaction offered 56 purchase related agent questions and was organized into 7 cycles of 7-10 questions that a user could run through with the agent. The 7 question cycles were displayed to the user on a category survey page leaving him the choice to run through the agent questions in any order he preferred and to the depth he deemed necessary. Within each question cycle it was ensured that with the help of a ’dialogue control box‘ (situated on the upper left part of the screen) users would be aware of the questions still to come in a question cycle and control for the questions already answered or skipped. Users were not forced to provide any answers. At the bottom of answer options to each question there was one graphically separated option entitled as ’no answer‘. Based on any number of multiple-choice answers provided by the participating shopper, Luci could be asked to calculate a Top-10 ranking of products.<17> From this ranking list, customers could then view a more detailed description of each product and enlarge its photograph (phase 3). The detailed product description contained a brief marketing text on the respective model displayed, the enlargeable photograph and a fact sheet summarizing major product attributes for each alternative. This phase closely resembled the current user driven style of electronic commerce environment. Appendix B1a gives an overview of the navigational phases the experimental participants encountered.

In the analyses presented hereafter, the communication phase with agent Luci and her recommendations will be considered as representative for agent-based information search, while participants‘ inspection of product details represents a typical form of manual search. With the three shopping phases orientation, dialogue and detailed product inspection, navigation resembled an offline store visit. The shopping process could be exited at any time and a purchasing decision could be made after the request for a product information page (in phase 3).


42

3.3.2 Store Manipulation

An important condition under which real-world online shopping takes place is that users‘ demographic identification data is often known to the host of a web site. Websites such as that of the infomediary Yahoo!.com<18>, for example, offer users the possibility to register with the domain. As a result, navigational behavior can be attributed to a person and online marketers are enabled to create personal profiles of their customers. As was discussed, many studies have revealed that privacy concerns of users oppose these practices. In order to create the same type of privacy-sensitive environment two manipulations were integrated in the store: First, agent Luci addressed a user with his or her first name, using the data that had been collected from candidates during registration. And second, participants were given the opportunity to provide their home address. Thus, after phase 1 where participants viewed all products one by one from a list, and just before phase 2, a html-page appeared on which shopping agent Luci introduced herself and her purpose to the user. All users had to pass this page and were given the possibility to leave their home address with the agent. No reason was given on this page why a user should do so, but two ’proceed‘-buttons were displayed on the bottom of the page: one labelled “save address, proceed“ and the second right below entitled ’no address specification, proceed‘. The user was left to decide whether to reveal the address or not without any sanctions.

Finally, another condition was integrated in the online store aiming to ensure extensive information search: no brand information was displayed on any of the products or product descriptions. The reason for this manipulation was that brand names were shown to serve as information chunks for consumers [Jacoby and Hoyer, 1981; Weinberg, 1981]. “Information chunks are information particularly relevant for the judgment of products and that are able to substitute or bundle a number of other information“ [Kroeber-Riel and Weinberg, 1999, p.280]. By avoiding brand names, it was ensured that all participants navigated under the same conditions and that superior levels of brand knowledge of some participants would not lead to uncontrollable ’short-cuts‘ by some subjects in identifying the right product.


43

3.3.3 Identical Store Design for Compact Cameras and Winter Jackets

In order to conduct analysis 2 (see table 2) it was vital to design the two store versions for cameras and winter jackets as similarly as possible so that differences in navigational behavior could be attributed to product nature and not to the store environment. The store therefore offered identical navigational opportunities and product display in its two versions including a similar quantity of products on offer, a similar number of attributes used to describe each product and an identical breadth of agent communication.

In addition, a considerable effort was made to provide for a similar perception of the agent dialogue in the two store versions. For this purpose, interaction was characterized and manipulated on three dimensions: First, the satisfaction with agent communication would have to be perceived as similar as possible for the two store versions. This put emphasis on the search algorithm used in the two stores (for a description of the algorithm used see Appendix B2). Secondly, the degree to which the agent dialogue would be perceived legitimate and important needed to be similar, in order to have people interact with the two store versions on the same premises. And finally, the way in which communication was organized in the store had to be the same as order effects have been shown to impact online navigation [Hoque and Lohse, 1999]. More detail on how identical store perception was ensured is commented on in section 5.1.2.

3.3.4 Development of Agent and Agent Dialogue

Abbildung 4: Image of Sales Agent Luci


44

Agent Luci deployed in the experiment as a female sales assistant was a selected option-based dialogue system. She was represented as a 3-D anthropomorphic and moving image (see figure 4). The reason why such a human-like interface agent was integrated into the system was that it was shown that visually personifying the interface (e.g. through a computer animated face) can lead to general social facilitation [Sproull et al., 1996].

In addition to this sociable side of the agent, the image was also used to draw users attention to specific details on pages, such as the permanent option to call for the Top-10 Ranking of products. The moving facial image was licensed from the company Artificial Life.<19>

Agent Luci offered consumers a catalogue of 56 questions to comment on purchase-related needs. Most of these questions were developed with the help of human sales agents selling compact cameras and winter jackets in a premium department store in Berlin. All of them were somehow linked to the purchase context, but many of them went beyond simple product attribute specification and also addressed ’softer‘ purchase concerns. The reason why softer sales aspects were integrated in the interactive system was to observe how far users would go in the revelation of personal information as a part of the information search process. Interest in users‘ marginal willingness to reveal information was also the reason why users were offered so many agent questions to answer. Seen that successful sales conversations in offline environments involve in average 3,3 questions answered to a sales agent [Haas, 2001], it was expected that the 56 agent questions integrated in the online sales dialogue would not be fully exhausted by most of the shoppers.

On the basis of group discussions among the researchers involved in IWA<20> all agent questions were characterized on two dimensions: First, each agent question was assigned to one purchase risk that it would primarily help to address, being either of functional, financial, social or psychological nature.<21> Second, each agent question was characterized as to the degree in which it would address the online user in person and thus intrude more or less in his or her privacy. Four privacy classes were


45

distinguished for this purpose: 1) non-private questions addressing specific attributes sought in the product (e.g.: How resistant do you want the fabric of the jacket to be?), 2) marginally private questions that referred to the consumer in person, but were also closely linked to product choice (e.g.: How important is the resistance of the fabric of jackets to you?) 3) relatively private questions looking into the usage envisaged with the product (e.g.: Where do you want to wear the jacket?) and 4) purely private questions that would somehow be related to the sales context, but be completely irrelevant for product choice. (e.g.: Where do you obtain your knowledge about fashion? in the purchase context for jackets). Appendix B3 gives a detailed overview of all agent questions as well as their respective assignments to risk and privacy classes. Here, more detail is also provided on the rules set to formulate questions and assign them to the classes in an identical manner (Appendices B4 and B5).

Finally, all agent questions were tested in an independent and separate study. Based on the judgment of 39 subjects (see table 2), they were rated as to their perceived legitimacy and importance in an Internet sales context. In addition, the difficulty and willingness to answer them in an online sales context were respected. Mean ratings are summarized in Appendix B3.

3.3.5 Pre and Post-Shopping Questionnaires

Before and after the shopping trip, all participants were asked to fill out a paper-and-pencil questionnaire which was identical for both camera and jacket shoppers. Questionnaire data was used to measure independent variables potentially explaining the behavior observed during the online shopping sessions. Most questions used were taken from earlier studies in information search and other literature sources. The pre-shopping questionnaire (see Appendix A5a) addressed demographics, budget constraints [Dowling and Staelin, 1994; Moore and Lehmann, 1980], self-confidence [Kiel and Layton, 1981], information consciousness [Punj and Staelin, 1983], Internet experience and e-privacy concerns [Ackerman et al., 1999] as well as product perception. Measurement of product perception included product knowledge [Srinivasan and Ratchford, 1991; Kiel and Layton, 1981], product experience [Punj and Staelin, 1983; Kiel and Layton, 1981; Moore and Lehmann, 1980], perceived


46

product risk prior to purchase [Bettman, 1975 and 1973; Cunningham, 1967] as well as perceived uncertainty to judge product quality [Weiber and Adler, 1995a]. After the shopping session, participants were asked to comment on the perception of the sales agent, encountered purchase risk, motivation to shop [Jacoby et al., 1978], perception of flow variables [Csikszentmihaly and Csikszentmihaly, 1995], ability to recognize brands by the product form as well as perceived legitimacy and importance of agent questions (analogues to the independent study mentioned above) (see Appendix A5b).

3.4 Procedure

When subjects arrived at the laboratory it was first ensured that everyone had a good understanding of the incentive scheme. In preparation that a subject might purchase (and win the lottery), everybody was asked to sign and hand in a Consent of Payment (Appendix A2) document. The Consent of Payment was necessary as the experiment organizers did not offer credit card facilities and also had no postal distribution service integrated in the online service. The Consent of Payment further supported the aim of raising participants‘ awareness of online purchasing consequences. Then, participants sat down and filled out the first questionnaire. When they handed in this first battery of questions they simultaneously received a paper-based privacy statement (Appendices A3a and A3b) what would explain data handling policies of the experimental online store as well as a description of the navigational opportunities in the store (Appendix A4). The privacy statement surprised participants with the information that log-files would not only be used for research purposes, but also handed on to a 3rd party, that did not wish to be named for the time being. Two different types of privacy statements were used: In the ’soft‘ privacy statement (type 1), participants were told that an industrial sponsor, a reputable European company, would receive all navigational data. Also, their rights according to the EU Directive 95/46/EC were stated in this privacy statement, including the right to know who makes use of the data, to view them and if necessary change or withdraw them. In the ’harsh‘ privacy statement (type 2), participants were told that their data would be handed on to an anonymous entity, and that it was not known what further use would be made of their data. Before entering the store, participants were required to sign this privacy statement and hand it in to the experimenters.


47

After all questionnaires and privacy statements had been handed back to the experimenters, a briefing was read out aloud to the group and final questions were answered (Appendix A1). The briefing contained information on the further experimental process and hints to the organization of navigation in the store as well as agent performance. In addition, the privacy regulation signed was further commented on, telling participants that it would not be in the interest of the experimenters “to collect dummy data“. They would therefore be expected to give truthful answers, because the search engine would not work adequately otherwise. It was added that the experimenters would “prefer a refusal to answer a question from an agent, than a lie“. After all, participants were given “the explicit opportunity to not answer agent questions“. The way this privacy briefing was formulated and read out aloud to the subjects, one goal was to minimize sympathy , or ’warmth‘ with the experimenters. The reason for this was that laboratory environments tend to make subjects feel ’secure‘ and behave more trustworthy than they would naturally do in a real-world context, the so called ’Hawthorne effect‘ [Mayo, 1933]. Generally, the goal of the privacy statement was to create a navigational context similar to the Internet where data is collected not only by the host server of a visited service, but also by third party servers (i.e. advertising companies).

Finally, people were asked to take their time shopping and not rush through the store remaining for at least 30 minutes in the laboratory. In order not to adversely affect the feedback of their performance, however, they were also told to remain no longer than necessary in the store, and to leave it once they felt shopping was completed. Employing this time-manipulated set-up some of the influence of time cost that is usually present when people surf and buy online was avoided [Hoque and Lohse, 1999]. This was done consciously, because if people had been given freedom in time there would have been many users with different personal time agendas leading to uncontrollable earlier break-ups. The aim was to avoid this, for in the current study it was more important to control model variables than to observe the absolute time investment users make to decide on a purchase.<22>


48

Once participants had received the verbal briefing, they started out for the online shopping trip. When they had finished they gave a sign to the experimenters who provided them with the post-shopping questionnaire. Once this questionnaire was filled out, participants left the lab. Outside the lab, the lottery and compensation occurred as well as verbal debriefing discussions with the participants. The whole process took about 1,5 hours per session in which ten participants were involved at a time.

3.5 Benefits and Drawbacks of the Empirical Research

In contrast to earlier studies in information search (in offline markets) the empirical findings of this study do not rely on self-reported activities, but are based on actual behavior observed. As a result, our empirical research does not suffer from selected-memory effects; consumers recalling only parts of their behavior which they can remember [Kroeber-Riel and Weinberg, 1999].

Another benefit of the empirical study conducted is that a ’pure‘ and instantaneous impact from different behavioral constructs on information search could be observed. External effects such as branding could be excluded. Through questionnaire data it was possible to explain behavior. A study based on log-file data from a real-world website only would have made the collection of questionnaires difficult. In addition, information search would have been impacted by the fact that product brands are displayed and that pages are loaded with a vast range of distracting content.

By using a sophisticated electronic advisor agent it was possible to win insights into people‘s dealings with this emerging type of technology and its relative importance in the information search process in comparison to today‘s user driven consultation of detailed product descriptions.

At the same time, the complexity of the experimental set-up implies a number of disadvantages. First, in comparison to psychological studies in consumer behavior the current experiment leaves room for many variables going unobserved. For example, some participants might have intuitively liked the image of shopping agent


49

Luci more than others. This perception of visualization may then have impacted behavior. Second, it was impossible to control for all pre-dispositions of participants. Perhaps some really came only for the 20 DM compensation and were ready to take a 1:10 chance to leave empty. Others may have really come to buy a product. As there are limits to what one can measure as influential factors there are limits to the explanatory power of the observations made. Finally, the sample size was limited to only 206 which is a very small basis to reliably interpret behavior in the way this was done with Structural Equation Modeling presented below.

Fußnoten:

<16>

In fact, there have been two more empirical treatments, the results of which are not reported extensively in this thesis. They involved the display of brands (vs. no brands for the current sample) and the availability of a physical channel for product inspection (vs. no channel existence for the current sample).

<17>

Prior to shopping, subjects were told that if they did not wish to communicate with the agent at all the ranking of products would be in random order.

<18>

See also (on 10.01.02): www.yahoo.com

<19>

(on 10.01.02) http://www.artificial-life.com

<20>

See acknowledgments

<21>

For more detail on the concept of perceived risk, see section 4.2.1.

<22>

Other studies that are based on conventional log-file analysis can do so much more effectively.


© Die inhaltliche Zusammenstellung und Aufmachung dieser Publikation sowie die elektronische Verarbeitung sind urheberrechtlich geschützt. Jede Verwertung, die nicht ausdrücklich vom Urheberrechtsgesetz zugelassen ist, bedarf der vorherigen Zustimmung. Das gilt insbesondere für die Vervielfältigung, die Bearbeitung und Einspeicherung und Verarbeitung in elektronische Systeme.

DiML DTD Version 2.0
Zertifizierter Dokumentenserver
der Humboldt-Universität zu Berlin
HTML - Version erstellt am:
Mon Dec 23 13:23:15 2002