Healthcare

Healthcare
What is Evaluation?
Is a systematic process that determines the merit or worth of programmes, products, processes, personnel, and/or policies.

The thing being evaluated (program, product, process, personnel, or policy) is referred to as the evaluand.

Evaluators seek to understand how the evaluand is operating, and the impact of that evaluand on organizations and community/societies.

Evaluators then inform relevant stakeholders of the evaluation results and make recommendations to them on how to improve the evaluand.

Evaluators work with stakeholders and subject matter experts to answer questions such as:
1. What are we evaluating (the evaluand)?
2. What is the background information and current situation of the evaluand?
3. What are the criteria and standards that govern the evaluand?
4. How does the evaluand measure up to the established criteria and standards?
5. What must happen in order to improve the evaluand?
The main difference between Research and Evaluation:

The main difference between research and evaluation is that:
Research is usually is designed to provide results that go beyond an individual program or project and can be generalised to other populations, conditions, or times. This places additional requirements on research.
Evaluation on the other hand, usually focuses on a situation, such as collecting data about specific programmes, with no intent to generalise the results to other settings and situations.

In other words, research generalises, evaluation particularises.

Thus Evaluation considers the results on the population being served by the program or project within the context in which the program is provided. Research asks whether the population has the same characteristics as other groups and whether results can be applied in other contexts.

In the evaluation field, we are making a value judgment about the meritorworth of an evaluand(programme, product or process). We find out what the situation is (i.e., the current state of the evaluand), what the criteria (milestones – what should the evaluand be doing) and standards (level of acceptability for the criteria) are for the evaluand, and then we judge the worth of the evaluand based on those criteria and standards.

Research, seeks to understand what the situation is, but then probes into why the situation is that way and how we can better understand the situation with regard to developing, expanding, and testing various theories (hypothesis).
Steps taken for an Evaluation
There may be many ways to tackle your evaluation project, but we like to think of it in terms of six basic steps.
1. Needs Assessment – find the gaps AND to engage stakeholders
2. Proposal – to describe the programme
3. Data Collection – Evaluation Design i.e. milestones, outcomes, activities
4. Data Analysis
5. Recommendations and Reporting
6. Lessons learnt – Follow-up
Activity: List similarities and differences between Evaluation research vs Other types of research

Both evaluation and research start with questions. Both are conducted using a logical plan. Both use similar methods to organize and report data. Key differences are related to focus, goals, the research question and the generalisability of the findings.

Evaluation Research Other Research
Purpose Focuses on a specific programme or problem
(whether to identify needs, improve some aspect etc ) Testing or Generating a theory (hypothesis)
Formulation of question Specific to the project General
Goal Improvement
(or to justify a programme – needs assessment justification) Increased knowledge relating to the hypothesis being tested
Setting (area of application) wide wide
Standards for assessing quality Evaluation on setting and data collection methods derived from the field Research setting and data collection methods derived from the hypothesis (theory)
Roles of different stakeholders Set goals
Set timelines
Define success of programme

No pharmaceutical companies!
No tobacco and no alcohol companies!

Researchers: design research
Community engagement is becoming important in Australian research
Publications Reports Journal Articles
Reports
Types of Evaluations: Internal and External Evaluation

Internal Evaluations External Evaluations
Purpose Program Improvement (Formative)

Data are used as part of programme planning and to assist in the process of revising and refining.

Formative evaluation is a continuous process, designed to provide ongoing information for regular decision making
Outcome Attainment (Summative)

Addresses the outcomes of a program –
It is a statement of the success or failure of a program.

Summative evaluation does not seek to change current practice, it provides information as a basis for deciding if a program should be continued, modified, or abandoned

Expertise Resides in the organisation Resides with the expert
Greatest Strength More access to data and resources. Know program and setting. More objective in assessing outcomes
Greatest Weakness Expertise and objectivity Access to internal data and resources. Unfamiliar with program.
Formative evaluation:
– Adoption (who is delivering the project/what are they delivering?)
– Reach (is the target group getting the information and accessing the project?)
– Implementation (what is being delivered, how is it being delivered is it being delivered as intended?)
Bottom Line:
 If you need to evaluate outcomes, especially for an external funder, an external evaluation may not only be optimal but also required.
 If you want to look for ways to improve an existing program, an internal evaluation may give you the information you need.
 If you want to do a needs assessment to determine areas for future growth, either would be appropriate.

Learning Activity
1. Read the attached Health for Life Program summary and answer the following questions:
a. What is the situation the program is trying to change?
b. Who are the specific target group?
c. What are some of the program’s activities?
d. What is the underlying program logic?
2. How would you go about evaluating this program? What outcomes, activities and milestones could you assess?
3. A formal evaluation of this program has taken place, the report can be found at http://www.health.gov.au/internet/h4l/publishing.nsf/Content/21D7AED124027F5DCA2571950002F31E/$File/HFL%20-%20Evaluation.pdf . Although it is long (134 pages!) have a look at section 1.3.1, 1.3.2 and 1.3.6 to see what methods they have used to gather their information.
a. Can you think of anything that could have been done better?
Also have a brief look at section 3 to see what specific factors they have looked at in their evaluation.

 

Place your order now and save 15% by using the following discount code:  welcome15 to enjoy this special discount, place your order within 2 weeks