More than a third of the responses to Lambeth council’s consultation on the Railton low traffic neighbourhood (LTN) in Brixton were excluded from consideration because of concerns about suspicious or duplicated responses.
The council said there had been “concerted attempts to manipulate the outcome”.
A report prepared by the consultants Systra for the council says that a large number of responses to the consultation on the Railton, and also the Oval to Stockwell LTN, “were identified as being suspicious in nature, or potential duplicates”.
Both LTNs are to become permanent.
There were a total of 4,554 responses to the Railton consultation, 1,626 of which were excluded.
Of a total of 3,697 responses to the Oval consultation, 302 were excluded.
The figures are in an appendix to a Systra report on the Oval consultation.
“As with all research data,” Systra says in the appendix, “it is good practice to check and review the data collected prior to analysis.
“This ensures that the data carried forward to the analysis stage is as clean as possible; allowing the analyst to have confidence in the data being used, in order to draw genuine and robust conclusions from it.”
“A large number of responses” in the datasets for the Railton and Oval consultations “were identified as being suspicious in nature, or potential duplicates”.
The appendix lists the criteria used to identify suspicious responses:
Responses with exactly duplicated ‘Start Date’ timestamps (i.e. surveys which were commenced at exactly the same time);
Responses which provided identical responses to the open-ended questions within the survey:
Questionable pairings of demographic responses. These included, but were not limited to:
Aged 18-24, but “wholly retired; and
Aged 65+, but on a government supported training programme e.g. apprentice.
Cases where over 85% of responses were blank, as well as those who listed ‘Prefer Not to Say’ for more than five possible answers (the appendix says that responses flagged through this check alone were certainly not grounds for suspicion).
After these initial checks, Systra shared the ID numbers of suspicious/potentially duplicated responses with the council.
To remain in line with data sharing regulations, the council in turn shared the IP (internet protocol) address, home address, and name of respondent, for responses marked as potentially suspicious, only with Systra.
This enabled the consultants to make a further check to discover if the suspicious responses were “true duplicates” by identifying responses using a duplicate IP or home address.
This check “provided an estimate that up to 1,756 responses in Railton, and 156 responses in Oval were potential duplicate responses,” the appendix says.
Systra acknowledges that it “is possible that multiple responses could have the same IP address and home address, and still be valid responses” – for instance, multiple members of the same household.
However, in these instances “it would still be expected that responses would not be identical”.
Systra and Lambeth council agreed not to exclude any responses based solely on them having the same IP address or home address.
The council later identified a further 286 responses in the Oval dataset which were considered as suspicious.
This was because they had been entered in “perfect alphabetical and chronological order”, with identical formatting to open-ended question asking for the first line of the respondent’s home address.
The appendix says that, as a result of these checks, the following exclusion criteria was applied:
Responses with either a duplicate IP address or duplicate home address and having highly similar or exactly duplicated responses to open-ended questions;
Responses with demographic pairings which do not make logical sense (wholly retired 18-24 year olds; 65+ year olds training as apprentices); and
The 286 Oval only responses that had been entered in perfect alphabetical and chronological order.
Where the same person had responded to the survey on multiple occasions, but their responses had not been duplicated, Systra used their most recent response for their answers to closed questions, “to prevent over-inflation of reporting to closed questions” and combined all of their separate open ended responses into one response so all written sentiments were still captured.
The appendix says that “this approach represents a ‘low risk’ option, to ensure as far as possible that any excluded record was indeed a ‘true duplicate’ response.”
The total number of cases excluded was:
Railton: 1,626 responses excluded (2,928 responses taken forward for analysis).
Oval: 302 responses excluded (3,395 responses taken forward for analysis).
Cllr Danny Adilypour, joint Lambeth council cabinet member for sustainable transport, environment and clean air, said: “The decision taken to make these schemes permanent followed months of monitoring and a consultation period which included street stalls, specific engagement with young people to hear their views, organised walkabouts with local tenants and residents’ associations and street audits conducted by pan-impairment disabled-led organisation Transport for All.
“All in all, these efforts saw well over 7,000 people engage with the consultations, sharing their views on how they feel the projects work and what they’d like to see on their streets in the future.
“Sadly though, the consultation period also saw concerted attempts to manipulate the outcome with over 1,800 fake responses to the survey identified.
“I am pleased, though, that the council was prepared for this eventuality and had learned from the experiences of other local authorities across the country who have held consultations on active travel schemes and saw similar attempts to distort the outcome of engagement with residents and businesses.
“It is crucial that councils scrutinise the data that they get to make sure it is robust, because it is incumbent on us to make sure decisions are made on a sound basis and for the benefit of our communities.”
