Considerations on the Evaluation Session of “Integrated Projects” (New Knowledge Instruments for FP6 "IP" ) Written by an Official Evaluator for LRE/EGOCREANET/Florence University Italy.
0) Evaluation Procedure ( 8 – Evaluation Points)
1st stage 5 days Briefings with all evaluators (about 30) and the staff (7-8) each evaluator 4 proposals (IP) . 5 Evaluators for each proposal . Last day Panel with all evaluators for a general review discussion and final ranking (8.30-20.30 ) 2nd Stage Hearings For 5 left proposals passing all thresholds. 12 evaluators 4 presenters for each proposal. 1 hour presentation, (45 minutes + 15 for questions) then 1 hour for discussion on the hearing. Then the last half day general final discussion with ranking !
1) The evaluation session is quite long. Proposals (IP) are now 200 pages long (sometime even 300) and the time left for reading and assessing it is not very long. The discussion with 5 evaluators and 1-3 moderators from the staff takes about 2 hours and it relates to all 6 points (no weight differences among them).
2)- Taking in consideration the first point “Relevance” the evaluators are invited, during the initial briefing , and even from time to time , to see if the proposal matches the objective of the specific programme; if it does, also from qualitative point of view, the mark should be 5 . However very often the evaluators forget this and relate most to the generally feeling of the proposal. It is in fact this the first mark.
3)The second mark is mainly related to the objectives of the proposal. The question is mainly related to the “exploitation” of the proposal.
<The presence of SME in the group is essential , the SME should play an “active” role, not be marginal ; The money they ask should be “important” , because they should play an active and essential part>.
Moreover for the point 2 is important to know what will be the exploitations after the project !! One proposal has been considered “perfect” from this point because it made a Table where they indicate very clearly 52 exploitations, who will be exploit it ,what , and approximately when after the end of the project. Then it is very important to precise such outputs of the project.
4) - The Third Mark: Money ! This is an essential point ! 15 millions euros is a sort of maximum threshold (for the entire project obtained by a sum of Full Costs and Added Costs !) This is independently from the number of participants (which is also a great concern !) . In some cases the evaluators do not feel “experts” and the common mark is 3 just at the threshold level in order to not be determinant! The average evaluator does not feel easy to assign a sum so important for a project where he does not consider himself an expert !
5)– Management
The structure of 3 levels, or with different levels is now considered standard (like in the Cowell proposal ) The main question has been : The coordinator is enough “authoritative” and get the clear adhesion of the consortium partnership to obtain the participants collaboration or help !
6)-Number of participants :However the best number of partners is around 20-25 . Absolutely no more ! A well balanced project with 53 participants has been attacked very heavily in the last meeting from the other evaluators (who did not read the proposal !) just for the number . Therefore the management should be structured in order to have no more than such number .Others can be subcontractors
7)- Note: Many critics for the absence of references , personal CV in the participants list, publications etc. which are not now specifically required now in the proposal. It appears only the organisation, the Company and this limits the evaluation on the capability to carry out the project.
The average evaluator does not feel comfortable without such references !
8)-Management
Professional organisations now write and assist the proposal Too many words, Phrases without meaning, etc. are common ! It is common feeling that the management should be carried out by a professional INTERNATIONAL co-organization !!