Journalists’ level of knowledge on empirical research and opinion polling: A study of Kenyan journalists

Introduction

Public opinion polling has become a common and important part of the democratic process not only in the developed world but also in the developing world. While public opinion polling is well advanced in the developed world, it is mostly in its nascent stages in the developing nations.

The increased role of opinion polls in society has raised the stakes on the rigor of the methodologies and whether journalists understand key concepts and aspects of empirical research. Brettschneider (2008) notes that given the important role the media plays in conducting opinion polls and reporting on the results, it is expected that ‘journalists should have the knowledge of experts in the field of polls’ (p. 482). Although it is expected that journalists should have adequate knowledge in opinion polling, most of them lack the basics on research methods (Brettschneider, 2008).

The purpose of this study is to assess the level of knowledge of political reporters in one sub-Saharan Africa country – in this case, Kenya – on certain key concepts and aspects of empirical research methods and opinion polling. This study also evaluates the attitudes and opinions of Kenyan media editors on the level of preparedness of political reporters when it comes to understanding the opinion polling process and reporting adequately on opinion polls. This study will finally assess the willingness of the political reporters to undertake continuous training on best practices in opinion polling.

Kenya was selected for this study because pollsters in the country were accused of manipulating poll results in the lead up to the chaotic 2007 general election in which over 1300 people were killed (Ndati et al., 2014). The media was, in turn, blamed for failing to adequately vet poll results from the pollsters and instead, publicizing questionable poll results that had reportedly been manipulated to favor one contender over the other. The Kenya National Commission on Human Rights (KNCHR), a government-sponsored agency, identified the media as one of the ‘key players that fueled the post-election violence’, with its poor coverage of opinion polls (Saturday Nation Team, 2008).

The allegations that the Kenyan media contributed to postelection violence through its poor handling of results from surveys led Parliament to enact a Bill that sought to regulate the publication of poll results. The Bill, passed in Parliament in 2012, was immediately assented as an Act of Parliament by the president. The Act states that anyone wishing to publish polls must disclose sponsors of the poll, the pollster, the poll date, population, number of participants reached, methodology, participants’ level of education, and margin of error (Republic of Kenya, 2012).

The topic of journalist knowledge of opinion polling is an important one, especially in new democracies or partial democracies like Kenya. Because a close presidential election triggers claims of rigging by opposing political camps thus leading to interethnic fighting, skilled journalistic reporting on concepts such as margin of error could help put the poll results into perspective for the public thus lessening the perpetual claims of election rigging and interethnic fighting that characterizes Kenyan elections. The reason that accurate reporting is important is because it sets expectations for the outcome of the election. When expectations are not met, trust in the electoral system declines, thus eroding democracy and threatening peace and stability. Furthermore, poor reporting harms the ability of citizens to hold electoral authorities accountable. These problems are even more severe in countries like Kenya with close elections and a history of electoral manipulation.

In addition, this is an important topic to examine because understanding the news reporters’ level of knowledge on empirical research should help colleges and universities in Kenya better understand the changes to make to their curricula in order to equip journalists with the requisite skills. Findings from this study will also help media organizations, pro-media training agencies, and donor institutions better assess the willingness of journalists to attend continuous training on opinion polling and reporting.

The media system in Kenya

The origin of the media in Kenya is traced to the 19th century when the missionaries and the colonial settlers started the modern Kenyan press (Ochilo, 1993). Mbeke (2008) argues that the colonial state, postindependence politics, dictatorship, economic liberalization, and information demand had an enormous influence on the development of the Kenyan media. The current Kenyan media is thus a result of political, social, economic, cultural, globalization, and technological forces.

The development of Kenyan media is categorized into four phases; the Colonial Era (1895–1962), the postindependence Kenyatta Era (1962–1978), the Moi Era (1978–2002), and the Kibaki Era (2003–2008; Mbeke, 2008). Instructive to note is that the four phases as categorized by Mbeke (2008) are named after the governments and presidents who ruled Kenya from the colonial days until 2013, which is indicative of the extent to which the Kenyan media system is closely tied to the country’s political history.

The authoritarian colonial government extended its stranglehold to the press and treated the media as an unnecessary evil that deserved close monitoring and control (Makali, 2003). Radio programs were censored since the inception in 1927 and the colonial government’s authoritarian policy on the media hindered the growth of not only broadcasting but also the nascent media in general.

Although the postindependence Kenyatta government was keen on shaping media policy around the urgent need for national unity and development, it also set out to manipulate and control the media for propaganda purposes (Kiambi, 2014; Mbeke, 2008). With the government being the largest advertiser for the two mainstream newspapers then, The Daily Nation and the East African Standard, the Kenyatta government was able to directly and indirectly control the news media by having it promote the government’s political project of nation building (Ogola, 2011).

Daniel arap Moi’s government attitude toward the media was shaped by key political events that started immediately he became president in 1978. The falling out of the political elite over presidential succession in 1978, the attempted military coup in 1982, the agitation for multiparty democracy in the 1990s and the economic recession that started in the late 1980s increased Moi’s government desire to control the media (Mbeke, 2008).

Moi responded to demands for political liberalization with stiffer restrictions on media freedom, harassment of media houses that dared go against his wishes, and detaining and jailing of journalists without trial. The introduction of multiparty politics in 1992 increased the demand for the liberalization and privatization of the media and with Moi presiding over a state in political and economic decline, media organizations were faced with three serious challenges; crisis of power, ownership, and resources (Ogola, 2011).

Although the reformist Kibaki government was proactive in developing pro-media laws and policies, it remained ambivalent toward the media. However, the Kibaki government created the Office of Public Communication in 2004 which is in charge of briefing the media on critical national issues, created the Media Council of Kenya (MCK) to set media standards and provide a mechanism for self-regulation, and oversaw the enactment of a new 2010 constitution in which media freedom is protected under sections 33, 34, and 35.

According to Maweu (2014), the forces of media liberalization have affected the Kenyan media both positively and negatively. While leading Kenyan media houses such as the Nation Media Group, the Standard Group, and the Royal Media Services have expanded their businesses by launching additional newspapers, radio and television stations under their respective brands, the conglomerates are using their vast resources to control the media environment, minimize competition, reduce costs, and attract advertisers (Maweu, 2014; Mbeke, 2010).

As private media owners aggressively pursue profits through various forms of advertising, Kenyan journalists are facing accusations of seeking to protect commercial interests instead of public interest, a move that ends up benefiting media owners and advertisers (Mutere, 2010). Television advertising spending in Kenya rose from US$341.9 million in 2016 to US$414.3 million in 2018, while radio advertising rose from US$342.1 million in 2016 to US$402.3 million in 2018. Internet advertising rose from US$132.9 million to US$180.8 million, while newspaper advertising went from US$126.4 million to US$132.5 million over the same period of time (PwC Kenya, 2019).

Besides media concentration and media commercialization, other challenges that have recently dogged the Kenyan media include media ownership and the professionalism of journalists (Ali, 2010). According to Simiyu (2014), the Kenyan political class has invested heavily in the media industry. In 2009, President Uhuru Kenyatta’s family acquired a controlling stake in Media Max Limited, which owns The People Daily newspaper, K24 TV station, and a string of radio stations, among which are Kameme FM, Milele FM, and Meru FM. Retired President Moi has an ownership stake in the Standard Group, which owns KTN News TV, Radio Maisha, and the Standard newspaper, while Deputy President William Ruto has a stake in a firm that controls Kass FM, Kass TV, and Kass Weekly (Simiyu, 2014). The government controls the majority stake in the state media, the Kenya Broadcasting Corporation (KBC) and the Kenya News Agency (KNA), which mainly report favorable government affairs.

Politicians have taken advantage of the liberalization of the media to acquire ownership which they can use to secure their place in politics (Nyamnjoh, 2010). However, according to Ireri (2015), the involvement of politicians in media ownership puts into question the objectivity of news in Kenya, a country where media owners are known to ‘influence the work of the journalists’ (p. 244). According to Nyamnjoh (2010), the ethnic identity of African journalists to a large extent influences their political beliefs, their framing of political stories, and the position they take on issues in the media.

In a study with journalists at the Nation Media Group in Kenya, Wasserman and Mwende Maweu (2014) found that ‘media managers, journalists, and shareholders have political ideologies and affiliations largely influenced by their different ethnic identities’ (p. 175). Ireri (2017) concludes that ‘ethnic-biased coverage of the election campaigns is one of the factors that triggered the 2007 postelection violence . . .’ in Kenya (p. 254).

Literature suggests that poor training of journalists in Kenya contributes to the deteriorating levels of professionalism in the field. Although a good number of practicing Kenyan journalists have some training in journalism and mass communication, journalism training programs in the country are faced with a myriad of challenges among them lack of resources to procure modern training equipment, out-of-date courses, unqualified lecturers, and low remunerated lecturers (Allan, 2014; Ireri, 2018).

The mass media and the communication sector in Kenya remain vulnerable to system-wide pressures due to ‘weak, irresolute and inadequate legal, regulatory and policy framework inherited from the colonial era’, and there is an urgent need for a comprehensive communication policy that addresses important issues such as ‘media ownership and control, programming and local content, education and training, and capacity building for community media’ (Mbeke, 2008: 7–8). Since the Kenyan media cannot be disentangled from the political, social, and economic dynamics, ‘its development will therefore rest largely on sound media policies and a supportive legislative media regime’ (Ogola, 2011: 91).

Political opinion polling in Kenya: An overview

Political opinion polling in Kenya emerged as a noticeable industry following the reintroduction of multiparty politics in 1992 (Hornsby, 2002). But it was after 2002 following the end of the repressive Moi regime that the industry started blossoming (Wolf, 2009). The election of Mwai Kibaki as president in 2002 paved the way for apparent lack of interference by the government on market research and political opinion polling.

Although opinion polls have increased markedly since Moi left power in 2002, political opinion polling gains more visibility during general elections when a few polling firms conduct and release results of various political contests. The Kibaki government’s acknowledgment of media freedom saw an increase in the reliance on opinion poll results for news by media organizations.

The first challenge to the reputation of opinion polls came during the hotly contested 2007 general election when politicians from the camps of two leading presidential candidates, incumbent Mwai Kibaki and his main challenger Raila Odinga, accused pollsters of having been compromised in order to manipulate poll results in favor of their opponents. Kibaki was declared the winner with slightly fewer than 232,000 votes leading to accusations of vote rigging and sparking ethnic violence.

After the postelection violence, a hue and cry over the conduct of opinion polling firms emerged with sections of the society demanding that they be regulated. Consequently, a Bill that sought to regulate the manner of the publication of electoral opinion polls was tabled in Parliament in 2011. The Bill was ultimately passed in 2012 and immediately assented by the president. It became an Act of Parliament under the title, Publication of Electoral Opinion Polls Act No. 39 of 2012.

Before the enactment of the Bill in 2012, Kenya had in 2010 promulgated a new constitution that paved the way for the creation of more elective posts, apparently rendering elections more competitive in the country. Some of the most competitive posts created under the new constitution include those of county governors and senators, with the governorships being the most fiercely contested.

In the run-up to the 2010 referendum on the draft constitution, three leading opinion polling firms – Infotrak, Strategic PR and Ipsos-Kenya – almost correctly predicted the outcome of the highly charged referendum. In a front-page story headlined, ‘Opinion polls correctly predicted public mood’, Kenya’s leading newspaper, The Daily Nation, noted that ‘though blamed, along with the media and politicians, for heightening political and ethnic tensions in the run-up to the 2007 general election, this time the pollsters correctly predicted the public mood’ (Nation Reporter, 2010: 1).

The outcome of the referendum showed that 67 percent of Kenyans had voted for the passage of the draft constitution, while 30 percent had voted against it. The last poll before the referendum by Infotrak predicted that 67 percent would vote ‘Yes’, while 26 percent would vote ‘No’. Strategic PR predicted that 66 percent would vote ‘Yes’, while 20 percent would reject the draft constitution. Ipsos-Kenya, then known as Synovate, had predicted that 58 percent would approve the draft constitution, 22 percent would reject it, and 17 percent would be undecided.

Public trust in the pollsters was, however, short-lived because, in the 2013 general election, the country witnessed another hotly contested general election between then prime minister Raila Odinga and Uhuru Kenyatta, who emerged victorious in that general election. Poll results during the campaign showed the election was too close to call, and politicians and activists in different political camps were back again at accusing pollsters of being used by their opponents to manipulate poll results.

Shortly before the 8 August 2017 presidential election that was later declared null and void by the Supreme Court of Kenya, opinion poll results from Infotrak and Ipsos-Kenya had showed that the election was too close between incumbent Uhuru Kenyatta and his main opponent, Raila Odinga, giving politicians from the two camps the basis for dismissing the poll results whenever they were not favored to emerge victorious. Although there has been an increase in polling firms in Kenya, only Infotrak and Ipsos-Kenya have emerged as the most visible political polling firms in the last decade.

Data journalism and research methods

Given the emergence of data journalism as a key skill among journalists, it is important to note where it is located in relation to research methods. Data journalism is a form of storytelling technique where traditional journalist working methods are mixed with data analysis, programming, and visualization (Nygren et al., 2012). Using an example of an election, a data journalist could rely on poll findings, past pool performance data, past voting behavior, and economic conditions to predict voting behavior (Solop and Wonders, 2016).

While data journalism should be treated as a core skill for journalists, it is important for students and lecturers of journalism to understand that working with data first requires that one understands how it was gathered. According to Burns and Matthews (2018), ‘journalists are expected to be increasingly familiar with, and to stay abreast of, the language and processes that gathering and representing data rely on’ (p. 95).

Howard (2014) notes that data journalism students need to understand the basic concepts and processes of data journalism that are grouped into five areas – data collection, cleaning, analysis, presentation, and publishing. The first step – data collection – is reinforced with exposure to areas of research methods teaching such as sampling methods, sample sizes, response rates, question-wording, sampling errors, and organizations commissioning the research. Although core areas of research methods such as the emphasis on understanding sampling techniques seem to precede key aspects of data journalism, the two areas are interdependent and knowledge in any one of the two areas is certainly a worthy addition to the skills of a journalist.

Inadequacies associated with opinion poll reporting

Although no empirical studies have been done on how Kenyan media or journalists have interpreted and reported on some key concepts in opinion polling, a textual analysis of broadcast content from local radio stations showed that the vernacular radio stations ‘were partisan and at times allowed their stations to be used as vehicles for spreading messages of hate on the part of politicians and community leaders’ (Somerville, 2011: 97). Somerville (2011) notes that the Kenyan mainstream press ‘failed to prevent the dissemination of party propaganda and the violent rhetoric of many political leaders’ (p. 90).

While a number of studies have looked at various issues relating to news media’s reporting on opinion polls, the main focus seems to be on the inaccuracies resulting from poor interpretation of poll results and failure to disclose enough information about opinion polls. One of the concepts from opinion polling data that are frequently misinterpreted by journalists is the margin of error, which Petry and Bastien (2013) describe as ‘the most misunderstood and abused mathematical concept that journalists have gotten their fingers on’ (p. 1). Failure to understand the correct interpretation of margin of error could lead to situations where journalists provide ‘fantastic explanations’ for ‘possibly meaningless shifts’ in polls (McBride, 1991: 190).

In a study of newspaper stories to assess the accuracy of reporting on the 2008 Canadian election, Petry and Bastien (2013) found that Canadian journalists did not have a ‘clear understanding of the ‘maximum’ margin of error that pollsters report, let alone the ‘true’ margin of error that they should be reporting’ (p. 13). In follow-up interviews with Canadian journalists, Petry and Bastien (2013) noted a widespread lack of understanding of the accurate definition of margin of error.

In a study of TV news during the 2000 US presidential election, Larson (2003) found that 47 percent of stories were ‘inaccurate because they misused the term statistical dead heat, claimed that one candidate was ahead when the results were within the margin of error or said that the results were outside of the margin of error when they were not’ (p. 66).

The inaccurate reporting showed that journalists had a major misunderstanding of what the plus or minus sign meant (Larson, 2003).

Although no studies have done an actual assessment of the understanding of margin of error among the public, Traugott and Kang (2000) projected that the actual knowledge of margin of error could be worse given that only 16 percent of respondents on a 1987 Roper poll gave an accurate answer about what margin of error meant in spite of 48 percent having claimed to know what it meant in self-assessment study. In order to help the public develop a better understanding of margin of error, journalists must first strive to gain an accurate interpretation of the margin of error concept.

According to Traugott (2008), inaccurate reporting on poll results by journalists can arise from ‘difficulties they have with statistical concepts; others come from lack of training in survey methods’ (p. 233). A clear understanding of important empirical and statistical concepts and survey methods is critical in helping journalists understand how to adequately write about data from surveys conducted by their media organizations and independent polling firms (Traugott, 2008).

The main goal of this study is to assess the knowledge of Kenyan journalists on opinion polling, and the following research questions are posed:

Methods

To gather necessary data in order to answer the research questions posed in this study, both qualitative and quantitative research approaches of data collection were utilized. To answer RQ1, RQ2, and RQ4, data were gathered with a survey of Kenyan political reporters. To answer RQ3, in-depth interviews were conducted with nine editors drawn from the print and broadcast media.

Measuring the level of knowledge

The level of journalists’ knowledge of basic concepts in research methods was assessed using 10 questions with a mix of multiple choice, and true or false response options (see Table 1 for a list of knowledge items used in the survey).

OPEN IN VIEWER

Table 1.

Knowledge questions and the percentage of Kenyan journalists who answered them correctly.

Knowledge question	% answering correctly	Mean score	Standard deviation
1. For a representative sample to be obtained successfully, you do NOT necessarily need an adequate sampling frame or a complete list of people in your population of interest	76.1	5.33	3.00
2. One of the major advantages of nonprobability sampling is that you can generalize results to a larger population	36.6	2.56	3.40
3. Sampling error cannot be computed with research that uses nonprobability samples because not everyone has an equal chance of being selected as a participant	36.6	2.56	3.40
4. The difference between probability and nonprobability sampling is that with probability sampling every member of the population has an equal chance of being selected	50.7	3.55	3.52
5. Word nuances and ambiguities affect the accuracy of research data	71.8	5.03	3.17
6. Which of the following types of sampling has the least likelihood of being able to generalize to the population from which it is drawn?	67.6	4.73	3.30
7. A poll report indicated that Candidate A had a significant lead over Candidate B. The poll report indicates that Candidate A is favored by a margin of 52–48 with a margin of error of plus or minus four percentage points (+/–4%). What is the correct interpretation?	74.6	5.23	3.06
8. Which type of sampling is more likely to be representative of the population?	45.1	3.15	3.50
9. A researcher who uses a sample of readily available and accessible participants such as students enrolled in an introductory mass communication course is using which type of nonprobability sampling method?	76.1	5.33	3.00
10. According to the American Association for Public Opinion Research (AAPOR), which of the following elements should a newsperson seek out before publishing results of a poll?	66.2	4.63	3.33
Cumulative mean score		4.43	1.50

Mean scores and standard deviation from each question item also reported.

Range of scale: 1–7.

Findings

Of the 71 journalists who took part in the survey part of the study, 62 percent (n = 44) were male, 26.8 percent (n = 19) female, while 11.3 percent (n = 8) declined to indicate their gender. In terms of the media house, the participants were 43.7 percent (n = 31) from newspapers, 12.7 percent (n = 9) from news television stations, 9.9 percent (n = 7) from radio stations, 7 percent (n = 5) from digital/online media, and 15.5 percent (n = 11) from one or two categories of the media houses. Another 11.3 percent (n = 8) refused to indicate the media house they worked for.

In terms of the education level of participants, 59.2 percent (n = 42) had a bachelor’s degree, 16.9 percent (n = 12) an associate’s degree, 9.9 percent (n = 7) a master’s degree, 2.8 percent (n = 2) a postgraduate certificate, 1.4 percent (n = 1) a doctorate degree, while 9.9 percent (n = 7) did not indicate their level of education. An assessment of the participants’ major in college showed that 35.2 percent (n = 25) studied journalism, 18.3 percent (n = 13) mass communications, 14.1 percent (n = 10) communications, 11.3 percent (n = 8) media studies, while another 11.3 percent (n = 8) were in the ‘Other’ category, meaning they completed a major that was not listed in the survey. Another 9.9 percent (n = 7) declined to indicate what they studied in college.

An evaluation of the number of years of work experience that participants had showed that 28.2 percent (n = 20) had 3–5 years of work experience, 19.7 percent (n = 14) had 6–8 years, 16.9 percent (n = 12) had 9–11 years, and 12.7 percent (n = 9) had less than 2 years. Another 5.6 percent (n = 4) had 12–14 years, while an additional 5.6 percent (n = 4) had over 15 years of professional experience. Another 11.3 percent (n = 8) did not indicate the number of years of professional experience.

Results of RQ1, which sought to assess the level of knowledge of political reporters in Kenya on basic concepts in research methods, are shown in Table 1. Reported in the table is the percentage of reporters answering each of the items correctly, and the mean and standard deviation of each knowledge question item. Mean scores were obtained after assigning one (1) to the wrong response and seven (7) to the correct response for each of the knowledge questions.

As shown in knowledge question 2 in Table 1, 63.4 percent of participants did not know that findings from a nonrandom or nonprobability sample cannot be generalized to the population. Another 63.4 percent of participants did not know that sampling error cannot be computed from data that were collected using a nonprobability or nonrandom sample as indicated in knowledge question 3 in Table 1. Results also showed that 49.3 percent did not get it correct that the main difference between a nonprobability sample and a probability sample was that a probability sample ensures that each member of the population has an equal chance of being selected as shown in knowledge question 4.

When participants were asked the type of sampling – between probability and nonprobability – that was more likely to be representative of the population, only 45.1 percent of participants identified probability as the correct answer as shown in knowledge question 8 in Table 1. Results also showed that 67.6 percent of participants were able to correctly identify convenience sampling as a type of sampling that has the least likelihood of being able to generalize to the population from among random sampling, stratified random sampling, and systematic random sampling that were provided as options in knowledge question 6 as shown in Table 1.

One of the three questions that participants scored better on involved the interpretation of margin of error. As shown in knowledge question 7 in Table 1, 74.6 percent of participants got it correct that both candidates were statistically tied in the scenario presented. Knowledge question 9 as shown in Table 1 sought to examine whether participants knew the differences between snowball, purposive, volunteer and convenience samples going by the provided example. Results showed that 76.1 percent of participants got it correct, marking it as one of the two questions that had the highest number of participants answering correctly.

To answer RQ2, tests of statistical significance using t-test and analysis of variance (ANOVA) were conducted to establish whether the reporters differed on their level of knowledge based on gender, level of education, the media house they worked for, years of professional experience, or their major in college. The tests did not show any significant relationship between any of the independent variables – gender, media house, years of professional experience, level of education, and major – and the outcome variable, knowledge.

RQ3 sought to explore the opinions of media editors on the level of preparedness of the political reporters to correctly interpret opinion polls and report on the same. Data showed a consistency of viewpoints from participants when describing their experiences with the political reporters’ copy on opinion polls. Editor A who has 15 years of professional experience with one of the leading daily newspapers in the country described his general experiences:

When I am editing a copy on a survey, I have to be extremely careful because most reporters are inclined to incorporate statistics and percentages that do not emanate from the survey they are writing about. It’s very worrying.

Editor B who is a news editor with a television station and has worked for a combined 20 years with various television and radio stations was categorical when asked for her overall evaluation of the political reporters’ ability to accurately report on opinion polls:

I can definitely say there is no expertise in our newsroom and other places I’ve worked when it comes to reporting on surveys. I have to spend an inordinate amount of time verifying opinion poll related news stories with the organization that conducted the poll.

Editor C focused his comment on the interpretation of the margin of error, a concept he said most reporters are confused about and were likely to get wrong:

Most reporters ignore the importance of the margin of error in the interpretation of survey data. In an ethnically divided country where elections are essentially a tribal census, journalists cannot afford to get the margin of error wrong.

Participants in the in-depth interviews went beyond describing the lack of know-how that characterizes reporting on opinion polls by Kenyan political reporters. Data showed participants had attributed the lack of knowledge in the basic research process and opinion polling to poor training at the college and university level. Editor D captured the reason for the widespread lack of skill in opinion polls thus:

You cannot expect the reporters to understand or master what they were not introduced to in college. Most universities do not offer courses in research methods or even basic data journalism courses. And those that offer courses related to research methods are not as rigorous and practical as they should be.

Using data from the survey part of this study, RQ4 sought to measure the level of willingness of political reporters to attend workshops or refresher courses on best practices in opinion polling. Results revealed that participants were very open to the idea of attending workshops or refresher courses in order to deepen their knowledge. The cumulative mean (M) score from the scale measuring the reporters’ willingness to attend the workshops was 6.44. The scale ranged from 1 (Strongly disagree) to 7 (Strongly agree). Table 2 shows the mean and standard deviation scores from each of the three items that were used to assess the reporters’ level of willingness. The measurement scale produced an alpha reliability of 0.80.

OPEN IN VIEWER

Table 2.

Items measuring willingness to attend a workshop or refresher course on opinion polling and best practices in opinion poll reporting.

Question item	Mean score	Standard deviation
I am confident that my colleagues would be willing to attend a workshop or a short refresher course on opinion polling and best practices in opinion poll reporting.	6.37	0.68
I would be thrilled at the opportunity to attend a workshop or a short refresher course on opinion polling and best practices in opinion poll reporting.	6.56	0.69
I know individual journalists who would readily welcome a chance to attend a workshop or a short refresher course on opinion polling and best practices in opinion poll reporting.	6.38	0.86
Cumulative mean score	6.44	0.61

Mean scores and standard deviation from each question item are reported.

Range of scale: 1–7.

Discussion and conclusion

The purpose of this study was to assess the level of knowledge of Kenyan political reporters on a few aspects of empirical research and opinion polling, the views of Kenyan editors on the reporters’ level of preparedness when it comes to correctly interpreting poll results, and the reporters’ willingness to attend workshops or refresher courses on best practices in opinion polling.

A look at the percentage of participants answering each of the knowledge questions correctly and the mean scores for each knowledge question, the participants’ level of knowledge on the questions can be explained as at best, average. Although the cumulative mean (M) score from the 10 knowledge questions was 4.43, it is also instructive to note that a 63.4 percent of participants did not know that findings from a nonprobability or nonrandom sample could not be generalized to the population from which the study sample was drawn. In addition, 63.4 percent of participants did not know that sampling error could not be computed from data that were collected using a nonprobability or nonrandom sample.

The significance of a random or probability sample in the generalizability of poll results to the population, and the relationship between probability samples and sampling error are both simple concepts that are at the core of empirical research and which every journalist reporting on opinion polls must seek to comprehend because as Brettschneider (2008) noted, ‘journalists should have the knowledge of experts in the field of polls’ if they are going to report accurately and adequately on opinion polls (p. 482).

According to Wimmer and Dominick (2014), the key characteristic that distinguishes the two types of samples is that ‘probability sampling allows researchers to calculate the amount of sampling error present in a research study; nonprobability sampling does not’ (pp. 89–90). A probability sample must be used when a researcher is seeking to support or reject a hypothesis or research question and generalize the results of the study to the population (Wimmer and Dominick, 2014).

In spite of the centrality of probability sampling techniques in empirical research, results from the knowledge questions also showed that 49.3 percent of participants did not get it right that the main difference between a nonprobability sample and a probability sample was that a probability sample ensures that each member of the population has an equal chance of being selected as a participant in the final sample study. Given that it is the responsibility of a journalist to interrogate polling firms to ensure their data collection process and subsequent results met the stipulated standards of an empirical research process, a reporter’s level of knowledge on empirical research, and associated concepts need to be at least above average.

Though 67.6 percent of participants were able to correctly identify convenience sampling as a type of sampling that has the least likelihood of being able to generalize to the population from among random sampling, stratified random sampling, and systematic random sampling, the 32.4 percent of participants who got this question wrong is still a high number given the rudimentary nature of this question. Although a convenience sample such as a group of students enrolled in a certain course can be useful in exploratory studies, it should be avoided in most situations ‘because of the bias introduced by the respondents’ proximity to the research situation’ (Wimmer and Dominick, 2014: 93).

Considering the central role that the concept of sampling plays in selecting participants to a study, a journalist must strive to acquire and retain a clear understanding of all types of sampling that fall under each of the probability and nonprobability sampling techniques, and the assumptions undergirding each one of them.

The low to average levels of knowledge on opinion polling-related concepts among this sample of Kenyan political reporters was corroborated by data from the in-depth interviews with editors who were clearly not pleased with the quality of reporting that they had seen emanating from opinion polls. The overarching picture of an unskillful reporter that the interviewed editors painted of the Kenyan reporter is worrying given that it is these reporters who are entrusted with gathering facts that are supposed to form the basis of their news reporting.

Indeed as noted by one editor to the effect that ‘in an ethnically divided country where elections are essentially a tribal census, journalists cannot afford to get the margin of error wrong’, skilled journalistic reporting on concepts such as margin of error could help put the poll results into perspective for the public thus lessening the perpetual claims of election rigging and interethnic fighting that characterizes Kenyan elections (Editor C, Kenyan Reporters’ Level of Knowledge on Opinion Polling, 2018, personal communication).

These findings on limited levels of knowledge on opinion polling among Kenyan journalists must also be interpreted in the context of the structures within which journalism operates as reviewed in literature on media systems in Kenya. Issues to do with government control of the media that date back to independence, ownership, commercialization, limited resources, and political power are factors that directly or indirectly denied journalists not only the much-needed knowledge on opinion polling and reporting but also the freedom to practice professional journalism.

Efforts by successive governments to control and limit the freedom of the media deprived journalists of the room to practice professional journalism and seek opportunities for professional development and growth. The current trend where media ownership in Kenya is quickly shifting into the hands of wealthy politicians keen on consolidating their power could lead to situations where the politicians owning the media seek to influence the work of journalists. Lack of knowledge on opinion polling also needs to be interpreted in the context of the larger issues of education and training of journalists in Kenya. With journalism training institutions lacking resources to procure the requisite training equipment and qualified lecturers, the opportunities for educating journalists on important aspects of their profession get quite limited.

As key stakeholders in the Kenyan media industry, universities and colleges must collaborate with industry players to create required research courses that will equip journalists with the requisite research and reporting skills. Media houses could also sponsor practicing journalists for short courses, workshops, and fellowships on opinion polling with a view to bridging the knowledge gap. After the training, media houses could dedicate a few in-house journalists to double-check the reporting of surveys and polls. Local and international partners keen on making a contribution in strengthening the capacity of journalists to accurately report on facts can consider expanding grants in this area.

This study has potential limitations, key among them that the data are from a nonrepresentative sample of political reporters. Therefore, the findings of this study cannot be generalized to the population of political reporters in Kenya. Although there is no sampling frame that exists of political reporters in Kenya, future researchers can endeavor to put together a complete list of the reporters and then get a representative sample from that frame. Future studies should also try and get a larger sample that includes reporters from various workstations across the country and not necessarily those based in the capital, Nairobi.

The current study gauged knowledge via self-reported level of knowledge measures. Although this study conducted qualitative interviews with editors, future research could follow up the survey study with in-depth interviews with the reporters to help identify unobserved heterogeneity in survey findings, as well as help, determine possible unknown, or overlooked variables. Another limitation is on the number of newspaper journalists (43.7%) that were surveyed for this study which is high compared with the number of radio journalists (9.9%) that took part in the study. Radio is a more popular and influential media in Kenya and the participation of radio journalists must be increased in future studies. The limitations notwithstanding, this study gives a portrait of the levels of knowledge on empirical research among Kenyan journalists and factors influencing what journalists know about opinion polling.