Supervising third-party control bodies for certification: the case of organic farming in Italy

Abstract

The organic food certification system in Italy, based on the EU legislation, relies on private control bodies (CBs) that are supervised by national authorities. The aim of this paper is to investigate whether it is possible to test for differences in controls outcome based on CBs’ behavior. To this extent, we use a multinomial logit model estimated on data corresponding to the results of the inspections in the year 2015 in Italy. The model controls for two fixed effects represented by the type of operator and a contextual variable, i.e., the region where a firm is located. Significant variability in CBs behavior could induce adverse selection in the choice of the CB by organic operators. In this vein, we discuss some implications in terms of policies and consequences for supervision in the conclusions.

Introduction

Public and private standards are widely adopted in global food safety governance. Their enforcement relies mostly on third-party certification bodies that certify producers’ compliance with standards. The certification process provides guarantees about unobservable product characteristics—such as the process through which they have been produced—to all actors of the supply chain, such as manufactures, retailers and food service operators, as well to consumers. The certification is generally issued through a label that implies compliance with specific guidelines, requirements or regulations. In this context, the role of the private sector is very large: for example, in the USA, the size of private food safety auditing has been estimated to be at least ten times the size of federal government inspections (Lytton and McAllister 2014). This is due to many factors including the size and complexity of the task that requires high specialized technical capacity. Third-party certifications are used also in many other contexts to monitor standards (Hatanaka et al. 2005; Raynolds et al. 2007) such as financial accounting (White 2010) and environmental regulation (Duflo et al. 2013).

Generally speaking, enterprises have a strong incentive for compliance because they do not want to risk losing the certification as well as their clients and reputation, but still standards and certifications are vulnerable. A list of limitations is provided by Powell et al. (2013). One of the reasons that can undermine food standards, including organic certification, relies on deficiency in enforcement by third-party certifiers (Fagotto 2014). The economic analysis of the conflict of interest is a structural feature of any system of private standards compliance (Lytton and McAllister 2014) in which the auditor is paid by the entity being audited. The same authors argue that auditing firms have often a close relation with the firm that they audit as they can also deliver advisory services. In the case of the organic food market, as in many other cases, the auditor is chosen by, paid by, and reports to the audited firm. This feature may imply a conflict of interest between reporting the truth and reporting what is beneficial for the client. The phenomenon, known as auditor shopping, can determine strong price competition in the audit market. Several studies have found that some suppliers consider certification as an obligation rather than as an intrinsically motivated quality management system (Walgenbach 2007; Albersmeier et al. 2009; Lytton and McAllister 2014). Hence, it can be implied these suppliers are not interested in the highest possible standard of inspection and choose the fastest, cheapest, and least intrusive third-party audit. At the same time, certifiers can seek to minimize their audit costs and also have a financial incentive to be re-hired. Reducing auditing costs may mean doing less strict controls and, therefore, having a greater chance to regain a contract. But reducing costs can also mean not to invest in the training of control staff, in inspectors rotation and other best practices. Furthermore, as each inspector is an agent of a larger certification company, it cannot be assumed that every certifier (agent) pursues the same objectives and the same competence as the certification company (respective principal) (Arrow 1984; Albersmeier et al. 2009). Therefore, the central task of the certification process—the reduction of information asymmetry—can be fulfilled only if the institutions in charge succeed in assuring a high quality of control (Albersmeier et al. 2009).

In the EU, all operators who produce, prepare, store, and import organic foods are controlled for compliance with the organic standard on the basis of Council Regulation (EC) No. 834/2007¹. The regulation relies on the control of the whole organic supply chain on a third-party certification system. In case of compliance, the control body (CB) or the control authority issues a certificate (documentary evidence) to the operator. This certificate assures the adherence to the underlying organic standard and it has the function of a quality signal. The certified firm can then use the EU organic logo.

Three different control systems are in place in the different member states (Fig. 1):

• A. System of approved private control bodies

• B. System of a designated public control authority(ies)

• C. System of designated public control authority and approved private control bodies

Fig. 1

The organization of the EU control system for organic food products. Source: INEA, 2013

The majority of the member states have adopted a system of private control bodies while five member states have designated public control authorities and four have a mixed system of designated public control authority and approved private control bodies. Competent authorities are responsible for approving and supervising control bodies and control authorities, by organizing audits or inspections of control bodies as necessary and, where needed, withdrawing approval of control bodies that fail to satisfy the requirements. The basic structure of the different control systems in the member states is the same. Control bodies have to demonstrate that their certification processes are in line with the general requirements for product certification systems as laid down in the standard EN ISO/IEC 17065:2012. This is done by accreditation², a procedure by which an authoritative body (either a public or a private accreditation body) gives a formal recognition that the control body is competent to provide inspection and certification services. The member states report information on the control system and its supervision to the European Commission on a yearly basis. Private CBs are supervised by national authorities in order to ensure objective, independent and effective controls. In Italy, this function is performed by the Central Inspectorate for the Control of Food Quality and Fraud Repression (ICQRF) under the Ministry of Agricultural, Food and Forestry Policies (MIPAAF).

An audit of the European Court of Auditors in 2012 (ECA 2012) recommended that competent authorities should strengthen their supervisory role over control bodies by applying appropriate documented procedures for approving and supervising control bodies, by promoting harmonization in the definition of infringements, irregularities, and corresponding sanctions, and by promoting identified good practices. Following this audit, several actions have been taken at the EU³ and member states level, in order to improve the harmonization of controls. Examples are the National Law in Germany (BMELV (Bundesministerium für Ernährung, Landwirtschaft und Verbraucherschutz) 2012) and the Decree no. 15962 (20/12/2013) of the Ministry of Agricultural, Food and Forestry Policies (MIPAAF) in Italy. The scope of this legislation is to standardize the controls, defining a univocal correspondence between the non-compliance and the sanction.

Figure 2 gives an overview of how the control system is organized in Italy. Regional authorities and the Central Inspectorate for Fraud Detection (ICQRF) are responsible for the supervision of certification bodies.

Fig. 2

Overview of the organization of the control and certification system of the organic sector in Italy

CBs’ audits in the organic sector include direct inspections of production operations (inspections) and a review of written records prescribed by the regulation. The literature on the evaluation of control procedure is growing in recent years (Albersmeier et al. 2009; Canavari et al. 2008; De Gennaro and Roselli 2008; INEA 2013; Zorn et al. 2012, 2013; Gambelli et al. 2018). These authors have shown significant differences between control bodies, in terms of organizational and operational characteristics, as the internal structure and the rules for the implementation of controls in the company. These studies also showed that such differences might influence not only CB’s performance, but also the credibility and efficiency of the certification system. Several authors (Gambelli et al. 2011; Gambelli et al. 2014; Gambelli et al. 2014; Albersmeier et al. 2009; Zorn et al. 2012, 2013) deal with the occurrence of non-compliance at the firm level searching for structural and economic determinants of firm behavior, in order to advise with concern to the improvement of risk-based controls. They conclude that, in order to increase their efficiency, controls should be based on the probability of non-compliance considering the factors that affect this probability such as previous behavior, the cost of compliance, the presence of mixed production, and processing systems among others.

The analysis of how the control and certification system works can provide useful results for improvement in its design, enhancing its complementarity with traditional public regulation (Fagotto 2014). In this article, we present a statistical analysis of the results of CBs inspections of organic food firms in Italy in order to provide some insights into the reliability of the organic certification system. We test if and to what extent the existing variability in the result of controls can be related to the control body and to the firm typology (type of operator) and to a contextual variable represented by the region where the audited firm is located. We believe that our results could provide some useful insight to public vigilance.

The paper is organized as follows: in the first section we describe the data and we present some statistics; then, we illustrate the model and the results of the estimations while in the third section we derive some conclusions in terms of policy recommendations.

Data and summary statistics

The analysis is based on the results of CBs controls on Italian organic firms in the year 2015⁴. The data belong to the archives of the Italian Ministry of Agriculture and concern the totality of controls performed in the year. We have reprocessed the original data collected by the CBs building a consistent database of all the organic operators controlled in Italy consisting of 58,796 records in 2015.

Each record in the database represents one operator and contains the following information: name and registered office of the certification company, name of the inspector, name and address of the client, date of the audit, type of activity, overall result of the audit, specific non-compliance, type of sanction, ands activity for which the sanction has been given.

The number of organic firms according to the type of activity is reported in Table 1.

Table 1 The distribution of firms by type of activity in 2015

According to the EU regulation and the Italian law, a sanction is related to the outcome of the inspection in case of non-compliance. It is classified as warning, suspension, exclusion, and suppression according to the severity of the non-compliance. A warning is issued for light non-compliance or “irregularity”. A serious non-conformity or “infringement” precludes the marketing of organic products. Corresponding sanctions are suspension or exclusion that refers to the prohibition to sell all farm products as organic respectively with short- and long-time effects and batch suppression that is the prohibition to sell as organic the specific product for which the non-compliance has been detected. The classification of non-compliances (Nc) and sanctions is depicted in Table 2.

Table 2 Classification of non-compliance and sanctions

Most of the Italian organic firms successfully passed the controls and did not get any sanction. On average, 18% of controlled firms reported only irregularities, normally relative to the completeness of the documentation and easily remediable or minor breaches that do not compromise the production process, such as the inadequate application of crop rotation, exceeding the permitted limits, while 5.4% reported severe infringements (Table 3).

Table 3 Distribution of organic firms by type of sanction in 2015

Most of the exclusion measures are determined by the failure to comply with the provision of precautionary suspension or repetition of non-compliance. Suppression measures refer to actions that compromise the qualification of the products but not the conformity of the production process and/or the self-control system on the production method and which do not have prolonged effects over time. They mainly refer to the issue of plant production—especially the use of seeds and conventional propagation material in the absence of an exemption and in the absence of the requirements for the granting of the exemption—and to the issue of product specifications relative to the presence of residues of products not allowed. In Table 4, the sanctions by type of operator are depicted.

Table 4 Number of sanctions by type of infringement and operator

Econometric approach and results

Model specification

Our analysis is based on a multinomial logistic regression model described by two logit functions (Hosmer Jr et al. 2013) as follows:

$$ {g}_1\left(\boldsymbol{x}\right)=\ln \left[\frac{\mathit{\Pr}\left(Y=1|\boldsymbol{x}\right)}{\mathit{\Pr}\left(Y=0|\boldsymbol{x}\right)}\right]={\beta}_{10}+{\beta}_{11}{x}_1+{\beta}_{12}{x}_2+{\beta}_{13}{x}_{p3}={\boldsymbol{x}}^{\prime }{\boldsymbol{\beta}}_{\mathbf{1}} $$

(1)

$$ {g}_1\left(\boldsymbol{x}\right)=\ln \left[\frac{\mathit{\Pr}\left(Y=2|\boldsymbol{x}\right)}{\mathit{\Pr}\left(Y=0|\boldsymbol{x}\right)}\right]={\beta}_{20}+{\beta}_{21}{x}_1+{\beta}_{22}{x}_2+{\beta}_{23}{x}_{p3}={\boldsymbol{x}}^{\prime }{\boldsymbol{\beta}}_{\mathbf{2}} $$

(2)

The dependent variable Y is the outcome of the inspection, i.e., level of conformity which has been classified into three different categories: compliance (outcome variable coded 0, taken as reference outcome), slight non-compliance or irregularity (outcome variable coded 1), and severe non-compliance or infringement (outcome variable coded 2). The (p + 1) dimensional vector x identifies the three explicative variables with x₀ = 1 and β the parameters to be estimated. A general expression for the conditional probability in the three-outcome model is:

$$ \mathit{\Pr}\left(Y=j|\boldsymbol{x}\right)=\frac{\mathit{\exp}\left({g}_j\left(\boldsymbol{x}\right)\right)}{\sum \limits_{k=0}^2\mathit{\exp}\left({g}_k\left(\boldsymbol{x}\right)\right)} $$

(3)

for j = 0,1,2

The independent variables utilized in the model are categorical and have been reclassified by numbers:

• The CB. In Italy, there were 11 CBs operating in the year 2015⁵. The certification market in the food organic sector is quite concentrated as 3 bodies retain a market share of 62.3%. CBs are authorized by the Ministry of Agriculture and can operate in all the Italian regions, although they might concentrate their activity on some regions.

• The region where the firms are located. We consider the region as a proxy of the agricultural activities that are practiced in a specific territorial context, considering the high level of specialization of Italian agricultural at regional level.

• The activity code. This variable describes the activity carried out by the firm subject to the control. With respect to the type of activity, on the basis of EU regulation, organic firms are divided into the following categories: producers, processers, mixed typologies, and importers.

More specifically, we test the following hypothesis:

H1: The result of the control depends on the CB performing the audit. We observe that CBs adopt different practices with regard to the way they perform the controls and we assume that it can influence the outcome of the audit. In fact, on average, in one year, the ratio between the total number of visits from the CBs and the total number of operators was 1.3 and each operator received at least one inspection. On the same vein, 14% of the operators, on average, received not announced visits but this figure ranged between 6 and 24% according to the CB. Another information that shows that control procedures are not fully standardized is the number of samples that are taken from the controlled firms and analyzed that ranges between 6 and 22% of the number of controlled operators according to the CB.

H2: The result of the control depends from the specific territorial and agricultural context for which we use as a proxy the variable “Region”, e.g., the administrative region where the organic firm is located. This may happen because, under certain circumstances, i.e., related to biotic or abiotic stresses, farmers may temporally not be able to respect the organic disciplinary and some productive systems may be more affected than others.

H3: The result of the control depends on the typology of the operator: producer, processer, mixed operator, and importer. Our hypothesis is that the probability of being sanctioned is higher in the case of agricultural producers as their activity is the most difficult to standardize.

The model is estimated using the data relative to one year (2015). Preliminary analyses for the models were conducted using adequate independence tests (V Cramer test and Pearson χ²; Table 5). As expected, the Pearson coefficient between the control body and the degree of sanction is highly significant (V = 0.1775 and Pearson χ² (24) = 3.7e + 03) (Table 5).

Table 5 Results of the independence test between control outcome and dependent variables

Considering that some CBs concentrate their controls in certain regions as depicted from the independence tests which show a considerable dependence between the region variable and the CB variable, we have compared five different models including all the variables or only one or two independent variables at the time. From the results (Table 6), we argue that the full model explains better the cases where the productive context is relevant as the results are “depurated” by the effect of the association between region and CBs. This consideration is also supported by the value of the BIC test as reported in Table 6.

Table 6 Multinomial logistic regression model—estimation results (models with regional fixed-effect dummy variables)

Estimation results

Table 6 shows the results obtained through the multinomial model specified in the previous subsection. The table concerns the outcome (Y = 1), i.e., the slight non-compliance and the outcome (Y = 2) “serious non-compliance”, i.e., an irregularity which generates a suspension, exclusion, or suppression, while the reference group is represented by the conformity outcome (Y = 0). Concerning the explicative variables, we considered a benchmark, by default, the Region Abruzzo, CB_1, and the “producers” typology.

The hypothesis that all the coefficients are simultaneously equal to zero is rejected by the LRχ²-test for both outcomes (Y = 1 and Y = 2).

We accept H1 as the variable CB is significant for most of the control bodies⁶. The risk of the audited firm to being sanctioned for a serious non-compliance (in terms of relative risk ratio (RRR)) is higher than compared to the benchmark (CB_1) for some control bodies as CB_7, CB_8, CB_9, and CB_10. These are the CBs for which that performed the highest number of unannounced visits and sample analysis. This result has a strong policy implication with regard to the need for supervising authorities to establish procedures in order to improve controls reliability, such as the rotation of control bodies and inspectors.

The hypothesis H2 concerning the effect of the regional context holds only for seven regions for which the RRR is significant. For the outcome (Y = 2), the RRR is significant and double with respect to the reference region as in Basilicata, Calabria, Emilia Romagna, Puglia, Sardinia, Trentino, and Veneto. The result is confirmed when we drop out the CB variable (model 3). Different outcomes of the control procedure at the regional level can be explained by the fact that—due to technical reasons—in some sectors might be easier or more difficult to comply with the rules of the organic standard. This is confirmed by results found in the literature in specific risk factors. For example, Gambelli et al. (2014) demonstrate that in Italy a diversified type of farming and some crops as permanent grasslands, cereals, and dried pulses have a higher probability of severe non-compliance.

With regard to the activity of the firm, the results show that firms that produce and process organic products have a significant and lower risk of non-compliance compared to organic agricultural producers, confirming H3. This result is, in our opinion, fully understandable for both outcomes given the much lower degree of standardization of the production process in agriculture with comparison to activities such as processing and marketing. With specific regard to outcome Y = 2, for example, agriculture production is much more dependent than other activities on biotic and abiotic stresses that can compromise not only productive results but also the correct application of the organic protocols.

Conclusions

Although much of the literature does not question the issues of credibility and impartiality that conditions the use of third-party certifications, variations in how CBs work can cause confusions for consumers and frauds (Fouilleux and Loconto 2017). Reasons for divergent behavior can stand in how CBs interpret the standards (Fouilleux and Loconto 2017) or in the practices that they put in place. The relatively low number of severe infringements (around 5% of total number of operators) suggests that the organic system in Italy is in line with the major requirements of the EU legislation on organic farming. Nevertheless, our analysis has confirmed, with regard to the Italian organic food sector, that there is an association between the audit outcome and the CB performing the audit. More precisely, we find that the probability to be sanctioned is higher from those CBs that make a higher number of unannounced visits or sample tests. Similar conclusions have also been found in Germany for food quality controls (Albersmeier et al. 2009, Bravo et al. 2013 and Zorn et al. 2012). Similar results for Italywhereobtained by Gambelli et al. (2014) that estimated the likelihood of detected non-compliances with regard to the adopted procedures of one CB in Italy. On the same vein, using data from the British Retail Consortium global standards program, Bar and Zheng (2019) found that manufacturers were more likely to choose certification bodies that they perceive to be more lenient.

Considering that certification is crucial to protecting the integrity of the organic system, we believe that our results can anyway provide some useful insight to supervising authorities in order to reinforce the risk-based approach to controls as required by the EU legislation as well as their surveillance activity. As suggested by other authors (Padel et al. 2010), further improvement of the organic control system could be made through improved supervision and prevention of both intentional and unintentional types of fraud. Basing controls and vigilance on risk analysis can definitively improve their quality while reducing their costs, i.e., increasing controls effectiveness. Standardization of procedures, as implemented in the last years—both at the national and at the EU level—is an important step in enhancing controls reliability in the organic sector. Rotation of CBs and inspectors on one side and further standardization of fees and procedures could be other steps in this direction that would enhance the creation level playing field between control bodies and reduce competition for clients on the basis of a less rigorous system. Third-party audits represent a significant cost for producers and other operators of the food system, but if well run they can also be beneficial to the system and contribute to increasing the food quality culture and to creating an effective food safety risk management system.