U.S. Department of Energy Energy Efficiency & Renewable Energy U.S. Department of Energy Energy Efficiency & Renewable Energy

Measure and evaluate performance at key points in the process

Measuring performance at key points in the upgrade process (e.g., assessments, conversion rates, and financing applications) has helped programs understand where their processes are working smoothly and where they are not. This information has helped them continuously improve their program design and implementation. To monitor progress, successful programs have combined information from their project tracking systems with customer surveys, information from call centers, and feedback from contractors and lenders to understand the customer experience. Make data accessible for program staff to track progress, identify successful strategies, and detect points of failure.

  • Enhabit, formerly Clean Energy Works Oregon, established an extensive process for getting customer feedback at key points in the program delivery process to evaluate customer satisfaction and better understand why some homeowners chose to undertake upgrades while others did not. The program identified seven points in the program delivery process to gather information through feedback surveys and phone interviews: application, assessment, bid, drop-out, financing, completion, and experience after 12 months. The program credited this kind of customer communication and feedback as one of the keys to its ongoing success.

CEWO Feedback Surveys

Source: Clean Energy Works Research Planning, Will Villota, CEWO, 2012 (Presented during January 19, 2012 Better Buildings Residential Neighborhood Program peer exchange call).

  • Boulder County’s EnergySmart program sent an online customer feedback survey to homeowners who had completed upgrades. Among other things, the customer surveys affirmed customer satisfaction and identified the opportunity for word-of-mouth marketing. Surveys found that the vast majority of the respondents would recommend the EnergySmart service to a friend or neighbor. The surveys also surfaced some weaknesses that the program resolved. For example, some respondents noted contractor’s lack of response and professionalism as an issue, which led the program to develop guidelines for professionalism and customer contact. Surveys also noted that the assessment report was long and confusing, leading the program to develop a new, customized report that was easier to follow and clearer about next steps.
  • Connecticut’s Neighbor to Neighbor Energy Challenge used qualitative contractor and customer feedback combined with quantitative data to evaluate how well its outreach efforts led to home energy assessments. When informal contractor feedback alerted program managers that relatively few interested customers were following through to have assessments conducted on their homes, the program analyzed project data and found that only around a quarter of customers who expressed interest in an assessment had completed one. To diagnose the problem, the program analyzed data to see how customers were acquired, how long it took to send leads to contractors, and how long it took contractors to follow up with customers to arrange for an assessment. Through qualitative analysis, the program found, among other things, that customers didn’t understand what they were signing up for and may have been unwilling to say “no” to young and enthusiastic outreach staff. The program also found that its staff wasn’t following up quickly enough with people that wanted more information. In response, the program improved its process for distributing leads to contractors (e.g., linking contractors to homeowners in 1-2 days), created a “receipt” for interested customers outlining next steps, and set up a system to call non-responsive leads after two weeks. With these and other steps, the program increased its close rate 35% in one month after changes were implemented.
See all tips from these handbooks:

Design a residential energy efficiency program that integrates marketing and outreach, contractor coordination, incentives, financing, and program evaluation to provide customers with the products and services they want through a customer-centric process.

Manage third-party impact and process evaluation activities by coordinating with evaluators, transferring data, and overseeing evaluation deliverables. 

Develop a detailed plan for launching and operating your program that integrates all program components into a process that is customer-friendly and efficient for contractors and other partners.

Develop a plan and metrics to evaluate the effectiveness of your marketing and outreach strategies. 

Establish an evaluation plan that will allow you to determine how your financing activities are impacting the market.

Establish metrics and measurement strategies for understanding whether you are effectively achieving your program goals and meeting your customers’ needs, while identifying areas that can be improved.

Ensure a positive customer experience with your program from launch through implementation over time.

Improve your program’s efficiency and effectiveness through regular information collection, assessment, decision-making, adaptation, and communication.