Housing & Supermarket site selection
There are some major shortcomings in the site selection system.
Although the selection criteria have been correctly chosen, the scoring
and weighting system appears highly unorthodox and problematic. This is
despite amendments to the scoring system in February 2013. We
recommend the use of a more conventional scoring system based on those used by
local authority planning authorities, and for the separation of the scoring
system design panel and groups of individuals scoring
potential housing and supermarket sites. The selection scoring documents should
be re-written in plainer English and be more transparent to people who do not
have a mathematical background.
Housing and Supermarket site selection scoring system has been designed for its
purpose by MNSG, and has been reviewed by various bodies including Wiltshire
Council, including the Community Engagement exercise, and substantial changes
have been made to the scoring systems since its first iteration in mid
selection criteria are aligned
with statutory requirements and
stakeholder consultation. There
are 5 broad categories for housing and
Location: maximum possible score 200
Access: max score 150
Buildability: max score 230
Conservation impact: max score 300
Environment: max score 250
possible maximum score: 1130
(for a site scoring with the maximum negative scores in all
categories seem appropriately
chosen and grouped.
will refer to MNSG Housing &
Supermarket Site Selection Scores Report v. 2.0 dated 11-Feb-13
http://goo.gl/6F9Ku, the Malmesbury Draft Neighbourhood Plan Vol 1 version 5
March 2013, and
feedback on previous versions in the MNSG Community Engagement
Response to Criteria & Sites Feedback document dated 12 February
from the Neighbourhood Plan describing the approach is included in Appendix 1 of
this paper, and an extract from the MNSG Site Selection Criteria Explanation 12
Feb 2013 in included in Appendix 2.
We note the
objections to the scoring system made by English Heritage in their letter to
Wiltshire Council’s Planning Dept on 14 February 2013 in response to the revised
Waitrose supermarket application, in which they suggest they:
“… may wish
to engage in a robust appraisal of the scoring matrix by which the current site
has been deemed the preferred location with the Plan, and to explore this in
more detail at Examination”. Point
system has also been challenged by Gleeson in their appeal against refusal of
planning permission for development on land south of Filands in a letter dated
13 August 2012 from their advisers, BDA Landscape and
in the MNSG Non-Community Consultation Review document dated 20 January
2013, in which Gleeson challenge the criteria and the criteria weighting. In
their statements Gleeson claim that “the evidence basis [for the scoring
system] is fundamentally flawed…”
and describe attempts to use the system to score the supermarket and
housing sites which generated greatly different results.
We will not
discuss the actual scoring decisions or results.
problems with the scoring system
The method used by MNSG is highly non-linear (criteria values 1,2 & 5
and weighting factors of 2, 5, 10 and 20) and chosen to "increase the
discriminability between sites". Most scoring systems used by government bodies
including Local Authorities for decision-making in procurement or planning use
simpler linear scales, e.g. a categorical system (e.g. allocating ++, +, -, --
or A,B, C, D) or a simple linear scale (e.g. 1,2,3,4,5) to score features of a
particular site (see list of example sites in Appendix 3).
In examples discussed in Appendix 3, a
matrix is constructed where the score (e.g. 1,2,3,4,5) is combined with a
weighting factor, also often linear (e.g. 1,2,3,4 or 5) to obtain a weighted
suggest their “logarithmic” scale
"may better represent the psychological scaling of the magnitude words" (i.e.
those used in categorical methods).
However most people look at numbers in a linear way - so they interpret a
score of 200 derived in this system as being twice as bad/good as one of 100 -
even though the actual differences between sites may be considerably smaller.
Such distortion can be useful in some applications - e.g. in identifying &
ranking risks - but that is not the case here.
With such a
nonlinear scoring system the output is highly dependent upon the values used for
criteria values and weights. For example, the weighting factors used (2, 5 10,
and 20) are arbitrary and lead to even more discrimination than say the set 1,
2, 5 and 10 often used in this type of scheme. With the present system any contentious
scores can contribute significantly to the overall
Quantitative assessment of what are essentially qualitative criteria is
difficult - as evidenced in the academic literature.
Categorical schemes rank options in a qualitative way (e.g. best ..
better .. worse .. worst) without assuming one is twice, ... five times, ... ten
times (20 times in the case of
MNSG's weights) better/worse that another option.
Such schemes tend not to differentiate strongly between options but still
provide sufficient information to enable ranking of them to be
The system gives higher numerical values to undesirable outcomes in each
criterion so that the weakest site option will have the highest overall
score. As with the non linear
scoring, this system is harder to understand for non experts.
Thus for supermarket location the theoretical Firestation site gets an
overall score of 20 (good) and M’bury PCT scores 200 (bad).
In many decision-making processes including procurement processes, it is
usual to score options, apply weighting, then convert the final result into a
the lowest an acceptable proposal or tender will score is between 40-50%, and
the highest scoring proposal or tender will win.
In the MNSG system we can only see relativities, and it is difficult to
define a range of scores that can be deemed unacceptable or acceptable, in
advance of the scoring process.
The language is sometimes difficult to understand, for
doesn’t appear in dictionaries, the usual word is “mitigable”, and even
this term is rarely used, there is a real danger that only the highly literate
and numerate will fully understand the MNSG
To ensure transparency and avoid possible criticism of the scheme being
self-fulfilling, there should be some separation between the individuals
creating the scoring system and those applying the scoring
As a result
of these weaknesses the current scoring system is vulnerable to legal challenge,
as we saw recently in the Gleeson appeal, and in English Heritage’s objections
the scoring approach with more conventional language, linear scales and
weighting, so that overall scores can be calculated as percentages and be more
easily understood by all stakeholders
material omitted example from Brent
A minimum acceptable score should be set
before re-assessing the housing and supermarket
2. Re-run the process with a larger group
of scorers including technical experts and local residents, each individual
scoring the proposals independently.
We could perhaps have two panels of 10 people, an “expert” group, and a
resident group. This will enable more genuine participation in the process, some
reduction of bias and would stand up better to legal challenge.
3. Consider the use of decision-making processes
such as Delphi panels, see http://www.economicexpert.com/a/Delphi:method.html
Malmesbury Neighbourhood Steering Group
What others are saying about our Draft Neighbourhood Plan Over 500 residents have already looked at the draft plan. These are the comments that they have made about it.
Our consultation runs from the 5th of March 2013 to April 26th 2013. These are the published comments received up to 08/03/13.
All comments below were submitted to this online feedback portal. We hold the identity and addresses for each submission and have decided not to publish them unless specifically requested by the person who submitted them.