Description
Both program administrators and home performance professionals are interested in delivering high-quality services to customers, reducing service delivery costs, and becoming profitable and sustainable. Even with extensive planning, every program will experience unexpected challenges. You can use these challenges as opportunities to identify ways to improve your program.
Do not wait until the end of your program to evaluate and improve it. You should assess your contractor engagement and workforce development activities on an ongoing basis, using information gathered as you deliver your program, including program operations data as well as feedback from customers, contractors, training providers, and technician trainees. Your evaluation plan identifies the metrics, data-collection approaches, and overall schedule for tracking your progress, but you will also want to adapt and respond to regular and ongoing performance data and feedback to support continual improvement in your program.
Use the goals and objectives that you have identified to make sure that your program’s performance is meeting the standards you’ve set. This provides an opportunity to find out where your activities are working and where they are not, and make adjustments as needed.
As you work to improve your program delivery, you should maintain close communication with participating contractors. In addition to helping you gather feedback, this lets you enlist contractors and other partners in finding solutions to problems. It also helps you maintain buy-in for, and minimize the adverse impact from, any needed changes.
This handbook provides guidance and resources to help you:
- Track program operations and external feedback from program partners, contractors, and customers
- Review and evaluate the collected data
- Share performance data and feedback
- Make improvement decisions with relevant partners and stakeholders
- Communicate decisions and changes to contractors.
You should coordinate your efforts to assess and improve the contractor engagement and workforce development component of your program with similar efforts to use feedback to improve other program components.
Step by Step
As you deliver your program, it is important to remember that while the plans and resources you have developed provide the critical foundation, you must adapt based on feedback you gather through implementation and due to changing market conditions. To do this, collect data not just for the formal evaluation of your program, but also for ongoing improvement of your contractor engagement and workforce development efforts.
As part of your evaluation plan, you identified a variety of data that you will collect, including project completions, customer satisfaction, and quality control metrics. You will need to track program operations data, customer feedback, contractor feedback, and workforce development partner and trainee feedback. These data, if tracked and monitored regularly, should allow you to gauge progress and make adjustments as needed.
For more on contractor engagement and workforce development objectives, metrics, and measurement strategies, see the Develop Evaluation Plans handbook.
Beyond formal evaluation metrics, you should also look for leading indicators that signal whether you are on track to meet your overall goals and information that can help you improve your program. Whether quantitative or qualitative, the data come from the following four primary sources: program operations data, customer feedback, contractor feedback, and workforce development partner and trainee feedback.
Program Operations Data
Some of your richest data will be gathered in the course of business, whether formally identified in your evaluation plans or learned through regular communications with program staff, contractors, and training provider partners. These data won’t be limited to high-level goals (e.g., projects completed, energy saved, or jobs created). You will also gather data on installation quality and issues, customer experiences, processing times, and other metrics. Take care to track the important data so that they can be easily assembled and evaluated. Your program can benefit from tracking as much data as is feasible. Examples include:
- Are program applications and forms submitted with complete and accurate information?
- What issues surface with energy modeling, if required?
- What are internal program processing times and reject rates?
- During quality control inspections, what problems are being identified?
- From a timeliness perspective, are projects being completed within any mandated windows for submitting rebates and final loan documentation?
- Are people who participate in training programs finding work with participating contractors?
You can anticipate some of these questions; however, you should also train staff to look for trends so that you can identify issues early and collect other information as needed.
As described in your evaluation plan, you will collect program operations data at regular intervals based on reviews of materials such as:
- Assessment and upgrade rebate applications
- Quality assurance inspection reports
- Contractor participation agreements
- Customer and contractor surveys.
For many of these data, you will have identified specific quantifiable metrics, so it is a good idea to establish monthly and annual review times when you can review the numbers to identify trends, spot deviations from your targets, and then determine where you might need to take additional actions to address issues or reward superior performance.
Customer Feedback
Ongoing feedback from homeowners on their experience with contractors is an important part of evaluating the overall customer experience. This feedback can also point to specific key areas where contractors affect a homeowner’s positive or negative experience. Showing contractors positive feedback can encourage them to continue in the program. This information can help you identify strategies and processes that are working or areas that need to be strengthened, including contractor training, corrective actions, or process changes.
Connecticut’s Neighbor to Neighbor Energy Challenge, for example, collected and analyzed data on contractors’ interaction with customers, and learned that there was a significant drop-off in completion rates if more than three days passed before the contractor called the customer to schedule. The program used that data to implement actions to improve completion rates, such as an automated reminder email once a lead was entered into the database.
Some programs choose to publish customer feedback or ratings about contractors (see the Maryland box below), while other programs choose to keep this information between the program and its contractor partners.
Maryland Home Performance with ENERGY STAR Offers Homeowner Reviews of Contractors
On Maryland’s Home Performance with ENERGY STAR website, homeowners can rate and review their contractors. Those reviews are published on the site as part of each contractor’s information page. Site users can search for contractors and sort the results based on homeowner ratings. The site also displays sample reports from a list of “preferred” contractors, some of whose own websites display distinctions such as “Selected as a Preferred Contractor in Maryland’s Home Performance Program.” From January 2011 through June 2013, the program completed 192 residential energy upgrades with a total average reduction in utility costs of 15% per household.
In developing your contractor engagement evaluation plan and associated resources to support gathering customer feedback, you’ve considered questions to ask customers and ways to ask those questions, such as phone or email surveys. While you’ll focus on customers who have proceeded with work, do not ignore those who have opted not to proceed or have dropped out at any stage of the process. Gathering their feedback can help you determine how you can work with contractors to attract more qualified prospective customers and to encourage more customers to complete energy upgrades. Resources to collect customer feedback include:
Enhabit Uses Customer Feedback and Contractor Incentives to Identify Opportunities for Improvement
Enhabit, formerly Clean Energy Works Oregon, is a non-profit “assembly of allies” that provides a one-stop shop for whole-home energy upgrades in regions throughout the state. Enahbit’s mission is to end energy waste and transform communities with home energy upgrades. To help ensure continuous program improvement, Enhabit regularly collects customer feedback to see how contractors can improve their customer service.
Customer Feedback on their Experience with Contractors
(Responses form program participants April 2011 through March 2012)
Source: Spotlight on Portland, Oregon: Making the Program Work for Contractors. U.S. Department of Energy, 2012.
Enhabit customers consistently rated their contractors as hard-working, respectful, and timely in surveys (see figure); they also noted several areas for improvement, including doing a better job of cleaning up after working. Enhabit also learned that customers were frustrated about the lag time between the initial home energy assessment and the upgrade itself. By reexamining the program’s incentive structure, Enhabit determined the lag time could be reduced by giving contractors $100 after the assessment and $300 when the upgrade was completed. This approach increased customer satisfaction overall and rewards contractors for both energy assessments and savings.
Source: Spotlight on Portland, Oregon: Making the Program Work for Contractors, U.S. Department of Energy, 2012.
Contractor Feedback
As you identified partners, made design decisions, and developed your implementation plan, you got feedback from contractors. To best engage and motivate contractors to help meet your program goals, continue that active dialog throughout implementation.
Contractor feedback will help you understand contractor experiences with the program and how well workforce development efforts are meeting marketplace needs. When you developed your evaluation plan, you identified questions for collecting qualitative feedback on your program from contractors, customers, and other partners. Regularly checking in with contractors, even informally, on these questions can help you course correct prior to any formal program evaluation.
For example, you will want to collect contractor feedback on the following questions:
- Has the program explained procedures to contractors in a way that was clear and easy to understand? Are program forms and applications easy to complete?
- Are there ways to improve data reporting processes between the contractor and the program?
- Do contractors have sufficient understanding of program incentives to accurately explain them to customers?
- Can the contractor-program interface be improved from the contractor’s perspective?
- Do the communication channels the program has established, such as regular meetings, allow contractors sufficient opportunity to provide feedback and suggest improvements?
- Have contractors expanded their hiring of new employees or improved employee retention, due to participation in the program?
Evaluation questions such as these will help you adjust and improve your processes and requirements as you implement them (e.g., fixing errors in forms or better explaining confusing procedures). These ongoing improvements can help sustain contractor participation in your program and help make it easier for contractors to be successful in delivering energy assessment and upgrade services to homeowners. Ultimately, this should contribute to better outcomes in the overall evaluation of your program.
You can use a variety of mechanisms to gather input and facilitate two-way communication. Many programs use a combination of the following options to solicit input:
- Host regular meetings or calls at times that are convenient, in light of contractors’ work schedules (e.g., 7 AM breakfast meeting)
- Work with contractor representatives (e.g., a trade association, individuals selected from among the group of participating contractors) to identify issues and discuss program improvements with your program managers
- Designate program staff, such as an account manager discussed in Develop Resources, as the main point of contact for contractor partners
- Survey contractors who have and have not been engaged with the residential energy efficiency program to obtain feedback on how the program is doing
For more information on soliciting feedback from contractors, see the Developing Ongoing Coordination and Feedback Mechanisms discussion in the Develop Implementation Plans handbook.
Workforce Development Partner and Trainee Feedback
As you track the workforce development metrics (link to WF.6 Develop Evaluation Plans) you identified, don’t just note the numbers of students trained and hired—you should also gather information on partner and trainee observations that reflect the effectiveness of your program in linking trained people to participating contractors. For example, you might want to investigate these questions:
- Are the program’s training opportunities relevant and helpful to contractor needs? Do they enable technicians to more effectively participate in the program (e.g. by improving their marketing and communication skills, improving the quality of upgrade work, etc.)? Are contractor training sessions affordable and offered at times that work for contractors’ schedules?
- Does the program’s workforce development and training offerings help students develop practical skills that they can use in the home performance industry? Are the skill sets for mentoring and training appropriate, or do they need adjustments?
- Are the program’s training options easy for trainees to take advantage of? Are trainings affordable and offered at times that work for trainees’ schedules?
- For programs that assign trainees to contractors, what feedback do contractors have on the training quality and the process for finding and hiring trainees?
- Has the training helped students find jobs? Of those trained, who is not being hired, and why?
You can collect feedback from workforce development partners and trainees through targeted surveys. Ask workforce development partners and trainees to complete surveys immediately after trainings, and consider a follow-up survey to find out how well the training is serving technicians in their careers. Include questions in contractor surveys that ask whether contractor training and program orientation helped contractor staff participate more effectively in the program, and what skills and experience contractors are looking for in new hires.
Temporary Labor Pool Eliminates Upgrade Bottleneck
One year into the launch of NeighborWorks of Western Vermont (NWWVT), the program’s weatherization contractors were struggling to keep up with the surging demand for home energy upgrades. At the same time, contractors were reluctant to hire more employees because of seasonal variability: demand for home energy upgrades in Vermont peaks in the fall and decreases in the spring and summer.
To help contractors find the technicians they needed to reduce project delays—which also keeps customers happy—NWWVT established a nonprofit temporary labor pool called LaborWorks @ NeighborWorks. Temporary installers were sorted into two tiers: those with minimal or no weatherization training and those with weatherization experience or certifications. When an NWWVT contractor requests an additional worker, a LaborWorks coordinator is given 48 hours of notice to match an available worker from the temporary pool with the contractor's project description (e.g., nature of work, location, dates of employment). As of 2012, the labor pool had employed ten technicians, with an average of three to five workers in the pool at any point in time. Many contractors shared their enthusiasm for the pool, and shared that the extra staffing helped them to reduce their backlog of projects in the community. Two contractors hired trained temporary workers after employing them through the pool.
To find out more about NWWVT's temporary contractor pool, explore the Focus Series interview.
Tracking data is the first step, but you have to review data regularly. Using the strategy that you created for reviewing your program data, you will want to determine whether performance metrics that come in below expectations are indicative of challenges with specific contractors or systemic or program-wide issues.
Many issues will be specific to individual contractors. These issues can often be addressed through regular training focused on customer service or through more in-depth mentoring (see the training and mentoring discussion in Develop Resources.
If many contractors are having problems, you should investigate further to determine whether the problems can best be solved through additional workforce training or whether you need to implement program design or process revisions for the program. (See the “Make improvement decisions with relevant partners and stakeholders” step below.)
After evaluating the information you collect, you will want to communicate data with contractors and workforce development partners to provide feedback on contractor performance, keep contractors apprised of how things are going, and seek contractor input on possible solutions to any issues identified. Use the strategic approach that you created to determine what information you will share, when, and how you will share it.
In Their Own Words: Improve Quality Through Performance Metrics
Source: U.S. Department of Energy, 2012.
Your quality assurance plan includes quality standards for work, processes for providing feedback to individual contractors, and corrective actions that will be taken when problems occur. You will want to follow these policies and procedures when conveying performance feedback to technicians and their companies.
Quality Control Procedures and Corrective Actions in Washington’s RePower Program
The RePower program in Washington’s Kitsap County, including Bremerton and Bainbridge Island, created a standardized process for quality control inspections. Home energy upgrades completed under the RePower program could be selected for quality control inspections, and were rated “Pass,” “Needs Minor Corrective Action,” or “Needs Major Corrective Action” based on the RePower Weatherization Specifications Manual.
Ten percent of RePower rebate applications were randomly chosen for quality control inspection, and RePower staff scheduled an appointment with the homeowner within one week of selection.
If a problem was found to require corrective action, the contractor was required to take that action at no additional cost to the customer. A serious problem, or repeated occurrences of an individual problem, resulted in a performance improvement plan for the contractor or suspension from the RePower program.
Along with the procedures in your quality assurance plan, consider the following rules of thumb when communicating feedback to contractors:
- For issues and feedback that are not specific to individual contractors, consult with your contractor partners to discuss what you have learned and the implications for the program. Engaging contractors and other partners to help solve problems will often lead to better solutions and help maintain buy-in for your program.
- For complex information or situations where you expect many questions or want to actively encourage conversation, you might use monthly coordination calls or quarterly contractor meetings to gather feedback and ideas from contractors.
- When the data pertain to individual contractors, particularly critical feedback, you can deliver those data directly through phone calls, email, or meetings with program staff (e.g., the contractor account manager).
- Encourage effective behavior and successful contractors in addition to your efforts to remedy problems.
- Many programs use recognition and awards to reward top-performing contractors and help motivate contractors to complete more energy upgrades and meet other goals.
You may find that contractors get overwhelmed by a high volume of program-related communication. It is best to present information concisely and with other communications (see “Communicate decisions and implement changes” step below), and then use targeted feedback for individual contracting firms to recognize accomplishments and address performance issues. You should also be sure that contractors do not experience a sudden drop in communication from your program. Use program communications to maintain their interest in and focus on the program.
Your review of program data and contractor and customer feedback may indicate the need to adjust your program’s processes and requirements. All programs can improve over time by making periodic adjustments to procedures and approach.
When discussing their reasons for hesitating to participate in programs, contractors often say that the requirements change too often, making it difficult to adapt. Even so, you should not hesitate to make changes to improve the customer experience, encourage contractor participation, and raise the quality and consistency of home energy upgrades delivered. It is generally best to make decisions about these changes after reviewing data, collecting additional feedback on the implications of the data, and actively engaging partners to seek solutions while still ensuring contractor buy-in.
A good quality assurance process lends itself not just to recording quality deficiencies, but also to using the data you’ve gathered to improve the work being delivered throughout the program. Below are some common types of improvements your program might make based on program data and contractor and customer feedback.
- Offer training opportunities to help contractors improve:
- If you see multiple contractors consistently leaving important energy efficiency measures off work scopes, you can provide additional training, either technical or sales, to encourage those upgrades to be installed more regularly.
- If contractors are consistently struggling to meet code requirements or adequately seal around recessed lighting fixtures, you can work with contractors to identify additional equipment or training solutions.
- Work with contractors to identify solutions to problems:
- If your review of contractor scopes of work and field inspection reports reveal common problems with energy upgrades, work with your partners to brainstorm and implement solutions to those problems. Your decision is then more likely to get the intended results and be accepted by contractors.
- For example, if water heaters often fail combustion-testing inspections, you can engage contractors to determine whether that is due to a diagnosis issue, difficulty getting homeowners to pay for correcting problems that emerge during remediation, or even misapplication by the program of the relevant standard. New York State Energy Research and Development Authority (NYSERDA) found that some technicians were using equipment that functioned at a different sensitivity level than the equipment the QA inspectors were using; the program was able to correct this problem after communicating with contractors.
- Make changes to participating contractor requirements:
- You may need to change the requirements for contractors to participate in your program in order to improve the quality of services delivered to your customers, to increase the pool of contractors participating in your program, and/or to address other issues. For example, it may be useful to adjust the requirements for company accreditation and/or certification of technicians who participate in your program, including continuing education requirements.
- Revise guidelines and requirements for contractor interactions with customers:
- If customers report that they are dissatisfied with their energy assessment or upgrade experience, explore why the customers are dissatisfied and work with your contractors to identify ways to improve the quality of the customer experience and the quality of the technical work. It may be useful to change your quality assurance and customer interaction procedures to prevent future issues. Consider introducing or revising your contractor code of conduct to ensure that technicians use courtesy and respect toward homeowners. You may also want to increase the rate of field inspections until quality issues are resolved.
Austin Energy Tackles Loan Preapproval Expirations
As contractors completed home energy upgrades during Austin Energy's early program delivery, the program carefully monitored customer demand. One unanticipated challenge that Austin Energy overcame was the expiration of loan preapprovals due to high demand for contractor services. Homeowners' loan preapprovals were expiring before contractors completed the energy upgrades. This situation was costly for the lender, burdensome for participating homeowners, and frustrating for contractors. Austin Energy responded by working with the lender to send out weekly notices to participating contractors to keep them informed of when their customers' loan pre-qualifications were due to expire so that they could prioritize those projects in their work schedules.
For more information on using data to improve your program, see the Develop Evaluation Plans handbook.
How you communicate decisions and implement changes is critically important. Frequent changes, changes without proper buy-in, or changes without adequate lead time can all undermine your efforts to keep contractors actively participating in the program. Refer to your evaluation plan as you create your approach for communicating changes. Be consistent with the standards you established for interactions in order to maintain credibility with your contractors. Some general principles for communications with contractors include the following.
Source: U.S. Department of Energy, 2014.
Tips for Success
In recent years, hundreds of communities have been working to promote home energy upgrades through programs such as the Better Buildings Neighborhood Program, Home Performance with ENERGY STAR, utility-sponsored programs, and others. The following tips present the top lessons these programs want to share related to this handbook. This list is not exhaustive.
A residential energy efficiency program’s success is dependent on the quality of work that contractors conduct in customers’ homes. Indeed, an in-depth examination of selected program strategies found that effective quality assurance and quality control programs provided a foundation for quality upgrades and were achieved through numerous program design and implementation decisions and follow-through. Many Better Buildings Neighborhood Program partners and Home Performance with ENERGY STAR Sponsors found that tiered and onsite quality assurance strategies, in addition to file reviews of upgrades reported to the program, worked well. Most programs use a tiered approach, in which a program inspects the first several upgrades completed by a new contractor and then inspects a specified percentage of subsequent projects. Onsite quality assurance is a useful strategy, both as a way of gathering feedback and as a training opportunity.
Programs conduct a broad range of verifications, including checking contractors’ certifications regularly, implementing a mechanism to re-check certifications, and verifying home performance professional safety skills (e.g., combustion training). In addition to inspections and feedback, some program also identified standards for ensuring quality work, including standards for technical work, for diagnostic tools and installed equipment, and for professionalism and customer service. Setting those expectations helped allow contractors to understand what was expected of them and better enabled them to help programs be successful from the beginning.
- In New York, NYSERDA uses a tiered approach for quality assurance. Inspection rates vary based on the contractor’s status in the program (see NYSERDA’s QA Procedures). The program inspects the first three projects that all contractors complete. After these initial projects, the program inspects 15% of a contractor’s completed projects, and at least one project annually. Customers may also request that field inspections be conducted within one year of the contractor’s work. If contractors have repeated QA/QC issues, NYSERDA increases the field inspection sampling rate, generally to 50% or more. If problems persist and are not resolved, NYSERDA sometimes suspends contractors from the program according to its QA procedures.
- The RePower program on Bainbridge Island, Washington, created a standardized process for quality control inspections. Energy upgrades completed under the RePower program could be randomly selected for quality control inspections, and were rated “Pass,” “Needs Minor Corrective Action,” or “Needs Major Corrective Action” based on the current RePower Weatherization Specifications Manual. If problems were found to require corrective action, contractors were required to perform the corrective actions at no additional cost to the customer. Repeated occurrences of an individual problem or serious problems resulted in a performance improvement plan or suspension from the RePower program. The program randomly selected 10% of their rebate applications for quality control inspection, and RePower staff worked to schedule an appointment with the homeowner within one week of selection.
- The NeighborWorks of Western Vermont program in Rutland County, Vermont, designed a quality assurance approach as a means to gather feedback and incentivize improvement. The program produced monthly contractor performance reports that compared contractor conversion rates, and then provided incentives to top performers. This approach was a productivity driver that encouraged contractors to make improvements to their business practices. During monthly one-on-one meetings, the program checked on each contractor’s client status list, made sure that no customers fell through the cracks, and gathered contractor feedback during the conversation. The program also set a timeline by which contractors must submit assessment reports to homeowners, with penalties in place for late reports. Using this approach, wait times dropped from four months to three weeks. See the Concierge Programs for Contractors webinar for more information. This approach has given contractors and the program the opportunity to improve over time.
- The Town of University Park, Maryland’s STEP-UP program worked to address variability in the quality of work that its contractors provided. The program approached this problem in two ways. First, STEP-UP issued a request for proposals for contractors that met specific performance benchmarks. From those proposals, the program then selected contractors with whom they had worked well in the past and began listing them as “preferred” contractors on their website. Ninety-nine percent of customers began selecting contractors from this list. Second, the program employed an energy coach for participating homeowners, to provide regular quality assurance of contractors’ work. The coach provided intermittent inspections at customers’ request, when they had concerns or when they chose to assist the program by allowing them to check on the contractors’ performance. The energy coach reviewed work proposals for scope and price; as a result, customers were reassured that they were getting the work they needed at a reasonable market price and therefore were getting fair value. By playing these roles, the coach gave customers assurance that they were receiving high value work from contractors and incentivized contractors to do quality work.
Many programs used the information they gathered through their quality assurance efforts to recognize contractors that deliver consistent, high-quality work. Rewarding good contractor performance can help you build trust, strengthen partnerships, and boost workforce morale. You can incentivize contractors to work for these awards by posting them on your website, announcing them at awards ceremonies or other events, recognizing them in newsletters, and encouraging contractors to post the awards on their websites.
- To improve contractor morale and work quality, electric utility Arizona Public Service (APS) and Home Performance with ENERGY STAR program sponsor FSL Home Improvements developed annual Contractor of the Year awards to recognize their top five participating contractors, given for the first time in early 2017. These contractors receive marketing benefits including a Contractor of the Year program logo for their website and additional marketing support. The program had been monitoring contractors’ work through quantitative metrics since 2012 and developed the quarterly scorecard as a tool to communicate contractor performance in 2016. These scorecards show how contractors compare to anonymized top and bottom scoring companies, based on their quality of measure installation, scope of work, customer satisfaction, and energy savings achieved. The program calculates each score based on performance over the past four quarters in an effort to avoid overly penalizing a contractor for any one insolated issue that they subsequently address. Not only do these scores enable annual contractor recognition, they also allow the program to give contractors regular feedback and increase contractor accountability. This comparative scoring has fostered friendly competition among contractors. The program has seen increased interest from contractors on how to improve their scores.
- Enhabit, formerly Clean Energy Works Oregon, singled out its contractors quarterly with honors such as the “James Brown Award” for the contractor with the most completed upgrades and the “Promoter Award” for showing the greatest job growth from one quarter to the next.
- The annual Charlottesville, Virginia, Local Energy Alliance Program (LEAP) “Blower Door Boss” award went to the contractor performing the most energy assessments while scoring the highest on customer surveys. The “Ruler of the Retrofits” title was bestowed on the company that scored the highest on customer feedback surveys and quality assurance reviews on home performance upgrades in Central Virginia.
- Maryland’s Be SMART program used awards and public recognition of accomplishments to help motivate home performance contractors that worked hard to realize significant energy savings. Be SMART gave awards to top performers that completed the most upgrades. The program presented awards for the greatest number of HVAC and home performance upgrades, the highest assessment-to-upgrade conversion rate, and the “Accuracy Award” for best rebate paperwork submission.
Early on, many Better Buildings Neighborhood Program partners focused on providing customers with a range of contractors to choose from, while providing contractors with access to customers. Customer feedback received by some programs, however, indicated that customers were confused or overwhelmed by the choices. A comprehensive evaluation of selected program strategies implemented by Better Buildings Neighborhood Program partners found that programs were more successful when they provided customers with lists of pre-approved contractors; however, offering long lists of contractors without differentiating their products and services often led to inaction. To help customers distinguish between contractors and choose a qualified one, many programs provide customers with information about contractor skills, quality of past performance, proximity, and other factors. Some programs matched individual contractors directly with individual customers.
Customers can provide valuable information about the quality of contractors’ performance, and this feedback can supplement other information, such as field inspections, used to differentiate contractors based on their performance. Many Better Buildings Neighborhood Program partners incorporated customer ratings into the order in which they list contractors online, to help future customers select a contractor. Some programs also used rankings to evaluate contractors, support disciplinary actions, allocate benefits, and identify retraining needs. Through this approach, contractors had the opportunity to improve their standing and reap the rewards when customers saw that they could be relied on to do high-quality work.
- On Maryland’s Home Performance with ENERGY STAR website, homeowners can rate and review their contractors. Some contractors choose to reach out to their customers to encourage them to provide reviews. These customer reviews, along with contractors’ accreditations and services, are published on the website as part of each contractor’s information page. Users of the website can search for contractors and sort the results based on homeowner ratings and by geographical location. Users can also narrow their results according to which contractors participate in the customer’s local utility rebate program.
- Efficiency Maine provided customers with a “Find a Residential Registered Vendor” locator on its website. This locator listed the services each contractor offered, sorted the list by distance from the homeowner, and differentiated contractors based on number of projects completed and customer satisfaction. All contractors were added to the list when they met the program’s requirements. The list was sorted by location closest to the customer and number of completed projects, and also noted what services the contractor provides. The website also listed questions a homeowner could use to interview and evaluate contractors, such as “How soon can you begin?” and “How quickly will my work be completed?”
- The Town of Bedford’s Energize New York program learned that selecting a contractor was the primary barrier for homeowners interested in home performance upgrades. The program addressed this challenge by developing a rating system to differentiate high- and low-performing contractors. Contractors’ ratings were calculated using a combination of customer survey results, the number of BPI certifications held by their technicians, and their number of completed upgrade projects. Some contractors were dissatisfied when they received low ratings, and in follow-up discussions, program staff reminded contractors that they would have an opportunity for their score to be updated quarterly and reviewed the scoring criteria. As a result, many of those contractors decided to improve their overall score. The program also set a minimum standard of completed projects (i.e., six completed projects over the last four quarters) for contractors to be included in the program. This narrowing of available contractors made it much easier for customers to select one without being overwhelmed.
- Seattle’s Community Power Works began matching homeowners one-on-one with certified contractors to create the best fit based on homeowner needs, contractor skills, and contractor availability. The program found that its past approach of suggesting two or three contractors led to indecision and that the potential price advantage of competition among these contractors was not an important factor in homeowner satisfaction.
- Programs should be transparent about the process of matching individual contractors to customers and ensure that all qualified contractors have the chance to participate in the program by competing for upgrade projects.
- While Community Power Works did not encounter any issues, programs should recognize that this approach can limit competition among contractors and discourage the growth of new contractors in the market. Most programs, including Enhabit, Austin Energy, Energy Impact Illinois, and many others, mitigate this by allowing contractors who bring their own customers to the program to keep them, providing an incentive for the contractor to market themselves instead of relying on the program to generate demand.
Even with the best contractor partners, a program may sometimes encounter difficulties that require remediation. Consistent with Home Performance with ENERGY STAR program principles, many Better Buildings Neighborhood Program partners discovered that they could address these difficulties by establishing contractor requirements to set standards for quality work, a transparent remediation process, and measures for dismissing underperforming contractors. They found that the key is to make contractor requirements clear from the beginning of your program. Contractor participation agreements and codes of conduct for interactions with customers can help ensure understanding of standards and provide a rule of thumb for when issues needed to be addressed. Not all contractors are equally skilled or customer-service oriented. These programs learned that, in order to preserve their reputation, they needed to be able to confidently recommend any contractor on their list. It is important to apply corrective actions as needed in response to problems and deficiencies, as well as a procedure to respond to serious or recurring problems such as probation or dismissal from the program. By setting the bar high and dismissing contractors that failed to meet program requirements, these programs helped ensure consistent, quality customer service.
- Efficiency Maine developed a Contractor Code of Conduct that contractors sign, stating that they will respect the homeowner’s property, minimize disruption to the homeowner, and leave the home in as good or better condition as it was found. It lists 15 things that contractors will and will not do relating to communications, onsite behavior, and work practices. To assure quality in the program, a minimum of 15% of upgrade projects are subject to random and/or targeted onsite inspections, covering the pre-installation, installation, and post-installation phases. Efficiency Maine’s Program Manual outlines clear procedures that program staff will follow in the event that the inspections reveal errors, omissions, or inconsistencies. The manual also outlines procedures for removing a contractor from the program’s registered vendor list for repeated failure to correct deficiencies.
- Omaha and Lincoln, Nebraska’s reEnergize Program furnished its contractors with an Energy Upgrade Contractor Protocol and General Scope of Work, which governs contractor work processes and customer interactions. This protocol was intended to serve as a supplement to contractors' technical training. It provided rules that contractors were required to follow to achieve customer satisfaction throughout the upgrade process and also outlined basic safety requirements. Topics covered everything from how to greet the customer to cleanup steps once the upgrade was completed. The protocol was an important tool for ensuring that all homeowners had a pleasant experience with the program through their interactions with contractors. It helped the program achieve over 1,300 residential energy upgrades over a 3 year period that included program launch.
- The Southeast Energy Efficiency Alliance Better Buildings Chapel Hill WISE program in North Carolina discovered that even though contractors might have met the required program criteria and had qualifying credentials, the quality of their work and their understanding of building science varied substantially. To address these issues, Chapel Hill engaged an external training partner that worked with contractors on the quality of their work and the implementation of quality control mechanisms to improve future work. The program developed and implemented a contractor probationary and debarment policy and corrective action plan. Under that plan, contractors were subject to a corrective process that included a preliminary review of concerns, probation, specific requirements to return to the pre-qualified list after probation, and dismissal from the program. This policy helped the program systematically approach the issue of alerting contractors whose work fell short of the program’s quality standards, and to dismiss contractors who were unable to improve the quality and consistency of their work.
Contractors are more likely to serve as program champions when the program engages with them throughout program design, delivery, and improvement. Your contractors are the primary contact points with your customers, and the quality of their interactions and services strongly influences how customers view your program. Many Better Buildings Neighborhood Program partners found that gathering contractor input during the program’s planning phase helped ensure that the program would create value for contractors as well as for customers. The programs built personal relationships with contractors by demonstrating interest in their business concerns and needs. Indeed, an evaluation of over 140 programs across the United States found that programs were more successful when they fostered relationships with their contractors and communicated frequently with them.
In Their Own Words: Engage with Contractors From Day One
Source: In Their Own Words: Engage with Contractors From Day One, U.S. Department of Energy, 2012.
By communicating regularly (e.g., via a monthly breakfast meeting, other outreach events) with a core group of contractors, programs were able to better monitor program implementation and receive suggestions for improvement. These programs elicited feedback from contractors about how customers perceived program offerings, as well as input about what was working and what was not for both contractors and customers. Some programs surveyed contractors to collect a regular stream of information about how program implementation was going and to get feedback before rolling out new offers or program design changes.
- NeighborWorks of Western Vermont maintained steady lines of communication with its network of contractors to help ensure that barriers to getting work done in a timely manner were identified early and that solutions were collaborative. The program held monthly one-on-one meetings with each contractor to review client status and progress and to identify any problems and potential training opportunities. The program also organized bimonthly group contractors meetings focused specifically on sharing new techniques or products. NeighborWorks used regular contractor communications, performance feedback, and contractor incentives and competitions to help contractors improve their assessment-to-upgrade conversion rates. By engaging contractors and including them from the start on any proposed program revisions or promotions, NeighborWorks was able to improve program delivery.
- Enhabit, formerly Clean Energy Works Oregon, program is charged with saving energy and supporting clean economic growth. Much of its success has come from engaging contractors in a continual learning and improvement process. Enhabit solicits feedback from contractors at meetings every two weeks and uses this feedback to guide improvements. With support from the Energy Trust of Oregon, a few contractors collaborated to create the Home Performance Contractors Guild of Oregon, which enables contractors to organize their opinions into a unified voice and have a more formal role in program and regional policy discussions. When Enhabit engaged a new financing partner, the program asked the Guild to examine the loan product and approval process. Input from the Guild helped ensure that the product was something that contractors would be able to explain and promote to customers.
- In Washington State, the Repower Kitsap program started in a region where the home improvement market was fragmented and under-developed. Contractors were initially wary of one another, tended to work only in their specialty, and often did not have working relationships with one another. The program established monthly brown bag meetings to discuss program goals and requirements and to gather contractor input on the program. The monthly meetings helped contractors get to know and trust one another and develop productive working relationships. Many contractors even shared leads with other contractors who specialized in the types of projects they could not or did not want to handle.
- The Long Island Green Homes program began consulting with contractors during program design and continued to do so as the program launched and began full service operations. The program established contact with a core group of contractors it trusted, meeting with them regularly to review program status and direction. In particular, the program made it a priority to engage with contractors when rolling out program changes, asking them about their needs, concerns, and current state of business. In this way, the program ensured that program offerings were adding value for the home performance industry and that program requirements were manageable for contractors. For more information on the Long Island Green Homes’ launch and other pilot programs, visit the October 2011 Better Buildings Residential Network Peer Exchange Call Summary.
Resources
The following resources provide topical information related to this handbook. The resources include a variety of information ranging from case studies and examples to presentations and webcasts. The U.S. Department of Energy does not endorse these materials.
This case study of Arizona Public Service (APS) and Arizona’s HPwES Sponsor, FSL Home Energy Solutions (FSL), focuses on their continuous improvements designed to elevate customer and contractor experience while boosting program cost-effectiveness.
This case study discusses the strategies Clean Energy Works Oregon's (now Enhabit's) used to actively engage contractors to make the program successful (e.g., balancing contractors' work priorities, enforcing quality standards).
This example home performance scorecard shows how a contractor compares to anonymized top and bottom scoring companies, based on their quality of measured installations, scope of work, customer satisfaction, and energy savings achieved.
Questionnaire for contractors participating in the Green Madison program about their overall experience, level of participation, training, and available resources.
Survey for consultants participating in Green Madison and Me2 programs about their experiences with the programs.
Participant survey sent to Me2 customers that have completed at least the initial Energy Advocate visit.
Survey for people who signed up to participate in the Me2 program for home performance assessments, but ultimately decided not to participate. The goal of the survey is to help improve services for future participants.
To deliver the most effective residential energy efficiency programs possible, NYSERDA implemented a quality assurance process to verify that projects meet all program requirements while maintaining healthy and safe conditions for the occupants.
Homeowner data collection survey created by RePower.
Courtesy of Clean Energy Solutions. This presentation provides an overview of topics related to building the workforce for energy efficiency programs, including market characterization, stakeholder engagement, training and certification, and community workforce agreements. It includes information on the experience of Clean Energy Works Oregon (now Enhabit) in Portland, Oregon.
This report provides an independent analysis of the job creation impact of DOE's investment in energy efficiency programs, from 2010 to 2013. The analysis calculates the job creation results that would have occurred in the Southeast, based on the prevailing economic conditions from 2010 to 2013, had DOE invested in sectors other than energy efficiency.
This report demonstrates the results achieved to date by the Southeast Energy Efficiency Alliance. It highlights the experiences of Consortium programs, their successes driving further investments in energy efficiency improvements, and the challenges that hindered their progress. It also details the infrastructure, resources, and opportunities that support the deployment of energy efficiency programming, and the approaches that the Consortium has found best suited to the region.
This report provides an independent analysis of the economic performance of SEEA's 13-city, U.S. Department of Energy-funded energy efficiency upgrade consortium from 2010 to 2013. It estimates the net impacts of SEEA's energy efficiency programs on the economy of the southeast region as a whole, and on the economies of the states with participating programs.
This webcast includes slides and information on programs' use of concierge programs to support contractors. It highlights two program examples: Clean Energy Works Oregon (now Enhabit) and Vermont NeighborWorks.
This summary from a Better Buildings Residential Network peer exchange call focused on implementing process improvements using lean processes, an approach of continuous improvement, use of Standardized Workforce Specifications (SWS) to improve quality, and contractor feedback tools. It features speakers from DOE, New York State Energy Research and Development Authority (NYSERDA), and Arizona Home Performance.
This summary from a Better Buildings Residential Network peer exchange call focused on innovative approaches to increase contractors' work quality through feedback reports and contractor ranking, decrease quality assurance costs through remote quality assurance, and improve contractor engagement. It features speakers from Consumers Energy, Enhabit, and DOE.
This summary from a Better Buildings Residential Network peer exchange call focused on quality assurance of energy efficiency services.
Achieving energy savings goals and improving customer and contractor satisfaction while staying cost-effective makes managing home energy upgrade programs challenging. DOE's Home Upgrade Program Accelerator is working with program administrators to identify strategies that overcome challenges and achieve better results. The Arizona Home Performance with ENERGY STAR program completed process improvements that improved contractor satisfaction and deceased quality assurance labor. Build It Green implemented software improvements to their utility program's online rebate applications portal to accelerate data processing.
This presentation describes strategies for outreach to energy contractors and auditors, including contractor incentives.
Presentation on five steps to building a profitable contractor base. The steps include sensible program design and administration, certification and credentialing, communicating with contractors, contractor requirements (business vs. trade), and training and sales support.
This sample email survey template, created by the Better Buildings Neighborhood Program, was designed for programs to develop their own survey of successful program participants in order to assess customer experience.
Sample phone survey template for program contractors.
This sample phone survey template for program drop-outs, created by the Better Buildings Neighborhood Program, was designed for programs to find out why applicants that applied to participate in a program ultimately dropped out.
This sample phone survey template, created by the Better Buildings Neighborhood Program, was designed for programs to use with applicants who have been screened out from participating in a program.
This document provides a menu of initial questions for a program administrator or implementer to build on and use in developing a real-time evaluation survey to collect qualitative data from contractors.
This toolkit was designed to help residential energy efficiency program managers identify resources and opportunities to help contractors, staff, and volunteers enhance their understanding of building science; sales and marketing; program offerings; and business development.
Concierge Programs for Contractors - They're Not Just for Consumers Anymore
This webcast includes slides and information on programs' use of concierge programs to support contractors. It highlights two program examples: Clean Energy Works Oregon (now Enhabit) and Vermont NeighborWorks.
Energy Efficiency and Conservation Loan Program Webinar Series: #4 Residential Energy Efficiency Deep Dive, Part Two
This webinar is the fourth (in a series of six) hosted by USDA Rural Utility Service (RUS) and focusing on the Energy Efficiency and Conservation Loan Program (EECLP). The second in a two-part series, this webinar shares best practices from the more than 40 competitively selected state and local governments who participated in the U.S. Department of Energy’s Better Buildings Neighborhood Program. This webinar focuses on data collection and continuous improvement, partnering with financial institutions, community-based outreach, and quality assurance of contractor work. It also features a case study from Jackson Electric Member Corporation about their audit tools, rebates and loans, tracking and reporting, and marketing and advertising strategies.
This handout summarizes the key lessons learned regarding workforce development contained in the Better Buildings Residential Program Solution Center.
This publication includes best practices for how to create a quality assurance plan and the key components that these plans should include.
This report explores the approaches and research needs identified in the Building Retrofit Industry and Market (BRIM) Initiative through in-depth discussion with residential energy upgrade experts including a discussion of Marketing & Outreach and the program/contractor interface.