Description
Assess your marketing and outreach activities to ensure they are helping to meet your marketing and outreach objectives. Use the marketing and outreach evaluation plan developed earlier as an opportunity to understand if your activities are working and, if they are not, to make program adjustments. The results of this assessment will inform improvement of your overall program design.
Do not wait until the end of a campaign to evaluate your marketing and outreach activities. As your program progresses, continuous assessment of marketing activities will help differentiate the marketing and outreach strategies that are successfully driving customers to and through your program from those that are not.
Make adjustments to your marketing and outreach plans whenever you discover an approach is not working. Improving marketing strategies and tactics based upon your assessments will be critical to ensuring that resources are focused on what has been most successful and achieving overall program success.
The following steps will help guide you in assessing and improving your marketing and outreach activities:
- Track progress using key metrics
- Collect customer and partner feedback
- Review and evaluate data
- Decide how to improve marketing efforts
- Communicate decisions to partners.
Step by Step
With your marketing activities underway, now is a good time to assess whether these activities are achieving the objectives you designed them to accomplish. Your evaluation plan will guide how to assess the success of marketing and outreach activities undertaken to meet your program’s objectives for encouraging home energy upgrades.
As your program progresses, an approach of continuous improvement will help differentiate the strategies that are successfully marketing your program offerings from those that are not. Do not be afraid to make adjustments to your marketing and outreach plans whenever you discover an approach is not working.
To assess and improve upon your marketing approaches, you will want to:
Monitor effectiveness of marketing strategies and adapt as needed
Tracking marketing and outreach objectives is the first step in assessing and improving your marketing and outreach activities. As part of your program design evaluation plan, your marketing and outreach evaluation plan will specify what metrics need to be tracked to gauge success. Most successful programs review these metrics in a dashboard on a monthly, weekly, or even daily basis.
As discussed in the evaluation plan handbook, some key metrics you might want to monitor include:
- Number of leads, assessments, and upgrades associated with specific marketing campaigns
- Cost per lead achieved through a specific outreach channel
- Conversion of leads to assessments and upgrades
- Percentage of customers taking advantage of incentives based on an outreach campaign
- Effectiveness of sales training or contractor marketing efforts, based on contractor success
- Amount of time it takes for a customer to receive a call back from the program after expressing interest during an outreach event.
Depending on how much data you collect, tracking marketing data may require more than simple spreadsheets, especially if you have multiple marketing strategies. For example:
- The Connecticut Neighbor-to-Neighbor Energy Challenge program used a customer relationship management database to track and compare the effectiveness of different marketing strategies.
Your overall program evaluation might include a more formal evaluation process, including a third-party review, that can provide insights into the effectiveness of your marketing and outreach activities. Coordinate with program staff who are conducting this evaluation to ensure that marketing and outreach-related feedback is solicited and shared.
Depending on the metric you are measuring, you may be able to analyze results immediately (e.g., number of homeowners that sign up for home energy assessments at an event) or not until later (e.g., increase in website visitors due to online advertisements). Be patient, but be prepared to review and analyze your data as it becomes available.
Enhabit Tracks Inquiries to Determine What Works
Promotional codes in your marketing materials that consumers reference when inquiring about the program will help track which materials and tactics are bringing customers to your program. Enhabit, formerly Clean Energy Works Oregon, used coupon codes like the one shown at right on marketing materials and asked homeowners to refer to the codes when they contacted the program. Enhabit also provided contractors and community groups codes to add to outreach materials to track the effectiveness of outreach efforts. Program staff said promotional codes helped make prospective participants feel like they had something special, and that they created a sense of urgency to motivate customers. Enhabit and its participating contractors also used the codes to track the efficacy of outreach materials.
Promotional Flier Created by Enhabit Contractor
Abacus Energy Solutions, LLC, an Enhabit contractor, developed this flier that included a promotional code to be entered on Enhabit’s website.
Source: Enhabit, 2011.
In addition to quantitative metrics, collecting qualitative information from customers and partners, such as feedback on your marketing tactics or outreach materials, can help you assess marketing and outreach efforts and make continuous improvements.
Customer surveys can help you evaluate which of your marketing activities and materials are working best, and which are not as effective. Your survey might focus only on marketing and outreach activities (e.g., a qualitative survey to see which messages, marketing methods, and outreach are working best to motivate customers), or marketing and outreach-related questions can be added to an overall program customer survey. Choose whichever method—or both—work best for your needs and budget.
The Survey Says…
Surveys can be a useful tool for collecting feedback from customers on the effectiveness of your marketing and outreach activities. The tools below provide information on what questions to ask which types of customers, from those who completed upgrades to those who dropped out of your program, so you can collect quality customer feedback that can help improve marketing and outreach efforts.
- The U.S. Department of Energy’s Program Evaluation Topics & Questions Library for Program Participants provides a good starting point for developing a survey that collects qualitative data from program participants. Following are some questions related to marketing and outreach that you might want to ask participating homeowners in a customer survey.
- How did you first hear of the project?
- Tracking the origin of customer entry into a program is essential to understanding what approaches are working best, and where expectations might not have been met.
- What were your motivations for responding to a given outreach approach?
- This can show which methods and messaging were most compelling. To limit the variety of responses this question could elicit, consider providing a list of different motivations and ask participants to rate them on a scale ranging from not motivating to significantly motivating.
- Did you have any difficulty accessing the program’s website, application forms, or finding answers to your questions on the program’s website?
- This question can be expanded to cover specifics but should get at the ease of using program outreach to complete various steps of the process.
- What factors motivated you to: a) sign up for the program; b) complete a home energy assessment; c) invest in energy upgrades?
- The answers to questions like this can be multiple choice, or you can ask participants to rate prepared responses on a scale ranging from not important to very important.
- The U.S. Department of Energy’s Example of Survey for Successful Participants provides sample questions that can be emailed to customers before or after they complete the upgrade process. Surveying customers who complete the upgrade process can offer insights into your marketing materials’ strengths, weaknesses, and room for improvement. Marketing and outreach-related questions you might ask in an email survey to customers who completed upgrades include:
- Which materials and activities motivated you to take action?
- Was the website helpful at explaining program offerings? Explaining the upgrade process? Inspiring action?
- The U.S. Department of Energy’s Example Phone Survey for Screened-Out Applicants includes sample questions to ask customers who initiated program participation but dropped out at some point. Customer feedback to these questions can reveal process or messaging improvements that might need to be made to your marketing and outreach. Questions you might ask in a phone survey to customers that dropped out of the program include:
- Did your experience interacting with the program’s systems, staff, or partners dissuade you from continuing?
- Was the program offer not enticing enough or not clearly presented?
- Was there a difference between how the program was marketed to you, compared to how you perceived the program after receiving more details?
See the Toolbox in this handbook for more survey examples.
Beyond solicited input, customer feedback may come to your program unsolicited, through emails, phone calls, or letters. A tracking mechanism that allows for input from a variety of sources, not just those you plan to collect feedback from, will make it easier to evaluate marketing and outreach activities.
Marketing and outreach partners should also be a source of feedback.
- Regular check-ins with partners can provide the opportunity for your program to collect this feedback.
- Contractor partners typically have the most face-to-face contact with homeowners, so they might be able to report back on what they hear from customers.
- Call center staff can solicit feedback as well since they are speaking directly with customers.
Cincinnati Assessment and Improvements Lead to Significant Increase in Home Energy Upgrades
Like many programs, the Greater Cincinnati Energy Alliance (GCEA) had a pool of homeowners who signed up to learn more about the program or to receive a home energy assessment. GCEA realized, however, that many of the customers did not continue beyond creating their online profile to complete their home energy upgrade. Instead of accepting these customers as losses, GCEA had its full-time energy advisor make phone calls to each of these customers to learn why they were not completing their assessments or upgrades. Of the customers the energy advisor contacted, 50% who completed assessments followed through to complete home energy upgrades. After implementing this energy advisor model, GCEA experienced a 40% conversion rate from home energy assessments to upgrades.
Once you have collected feedback and tracked metrics identified in your evaluation plan, the next step is to review the data and customer feedback collected to understand which activities and materials are working, and which are not. Take the time to review the data and feedback received against these indicators of success.
When evaluating customer feedback, look at how customers and potential customers reacted to your program’s marketing materials and offerings in the context of your marketing and outreach objectives. Your staff and partners might love a marketing tactic that was used, but if it did not motivate customers to take actions outlined in your objectives, it did not achieve success. Evaluating data is crucial to helping you assess what is working and what is not and enables you to make informed adjustments as needed.
You might want to schedule the review and assessment of marketing and outreach data to coincide with review and assessment of data from other program components (e.g., Program Design & Customer Experience, Financing, and Contractor Engagement & Workforce Development) so that you can make the most comprehensive assessment of successful approaches and identify options for improvement where needed.
Two Bills, One Problem
Energy Impact Illinois created an award-winning video series, entitled “A Tale of Two Bills.” Unfortunately, the video series did not bring in any customers. Consequently, Energy Impact Illinois shifted their marketing focus to a more grassroots campaign.
Shifting to the grassroots approach, the “Two Bills” were still used to help establish the program’s brand among a broad audience, but new videos included a clear call to action. By combining a grassroots approach with broader marketing efforts that included the “Two Bills,” the program achieved more than 3,700 single-family home energy upgrades.
Source: Energy Impact Illinois, 2011.
Be prepared to review and evaluate your data, and make marketing and outreach adjustments as needed on a regular basis to ensure continuous improvement of your marketing and outreach efforts. Many successful residential energy efficiency programs achieve success by making marketing adjustments mid-stream. Initial assumptions and assessments about target audiences’ interests, needs, and responses do not always hold true, even when research helps inform the process. Market environments also evolve over time. Do not be afraid to reconsider overall marketing strategies, individual tactics, or materials that are not helping to achieve your marketing and outreach objectives.
Mid-Stream Adjustments Give Chicago a Boost
Energy Impact Illinois (EI2) methodically planned out each step of its residential energy efficiency program. Staff spent months conducting thorough market segmentation research, building a polished website, developing catchy advertising, and creating what they thought would be a beneficial loan product. What they did not do was talk directly to their customers. As a result, their marketing and outreach effort did not have a very successful start.
“The market segments, website, advertising, and loan product would have been more effective, or perhaps unnecessary, once we better understood our market through on-the-ground outreach,” shared David Haeg with CNT Energy, which served as the implementation agency for Energy Impact Illinois. “Plus, if we built a small loyal core of happy customers immediately, we would have had more compelling stories to tell the media and future potential customers.”
Mr. Haeg added that programs should attempt to follow a methodical approach to marketing, but cautioned that programs should not wait too long before testing their approach. “[Waiting] can easily result in teams planning and developing materials for far too long and consequently waiting too long to get into the field to see what actually works.”
Deciding to use face-to-face contact to build neighborhood trust, EI2 focused on door-to-door canvassing as an outreach strategy to encourage consumers to sign up for energy assessments. This strategy, however, neither targeted specific types of homeowners nor garnered serious interest in home energy improvements. Canvass volunteers found those who actually answered the door were more interested in the free giveaways they were offering.
House Party Model
Source: Energy Impact Illinois, 2013.
The lack of sign-ups for energy assessments caused EI2 to again revamp its outreach strategy, opting for a house party model. Trained field staff and contractors conducted an abbreviated energy assessment of a home with the homeowner and a group of their friends and neighbors. Over the course of a year, the program organized 652 house parties, and 3,110 people attended. More than 2,500 of these attendees signed up for assessments on the spot or called EI2’s call center to host a house party—a conversion rate of 82% from house party attendance to assessment sign-up. Read more about EI2’s mid-stream marketing adjustment in the U.S. Department of Energy’s Focus Series Interview, “Energy Impact Illinois Learns That Parties Sell Upgrades”.
Source: Energy Impact Illinois Learns That Parties Sell Upgrades, U.S. Department of Energy, 2013.
The data you gather on how your marketing activities and materials are being received among target audiences will be critical to both internal decision-making and partner contributions. Communicate your marketing data where appropriate and your decisions made about mid-stream improvements or continuous improvement, both internally and to partners.
You can communicate marketing and outreach decisions in reports, memos, presentations, email, face-to-face meetings, or conference calls with your staff and partners. If you already have a regular check-in method established with your partners, use it to update them on marketing and outreach decisions. In your updates, communicate the data and feedback from partners and other information that influenced your decisions, as well as planned mid-stream improvements.
Community Power Works Heats Up Demand With Targeted Marketing
Community Power Works (CPW) of Seattle, Washington, decided to focus on owners of oil-heated homes as part of its spring marketing strategy. Customer feedback from the program’s other campaigns helped the program determine that direct mail letters were the best way to communicate with this audience. The first mailing was successful at getting oil-heated customers to sign up.
The first letter was sent in April 2012. More than 700 CPW customers who signed up for upgrades from April to August reported mailings as the way they heard about the program. Additionally, in the 11 months prior to the first mailing, only 20% of CPW’s upgrade projects involved oil-heated homes. However, during the six months following the mailing, 50% of the homes that started the upgrade process were oil-heated. Among those homes, nearly 75% switched from oil heating to high-efficiency electric heating or high-efficiency electric heat pumps by mid-December 2012.
Not quite ready for the surge in demand that resulted from the mailing, CPW did not have enough qualified contractors at first to deal with the increased heating, ventilation, and air conditioning (HVAC) work. CPW quickly worked to increase their contractor base and engaged HVAC contractors in Seattle known to turn around projects very quickly.
CPW learned that program staff needed to inform contractors that an influx of customers with outdated oil heaters could be expected soon after each mailing, or risk not having the capacity to respond to homeowners who were ready to switch to more efficient models. CPW staff alerted contractors before subsequent mailings were sent to oil-heated homes and when contractors could expect evaluation sign-ups to spike. They also communicated the program’s expectations for turnaround time so contractors could clear schedules to receive the program’s leads.
For more information, read the U.S. Department of Energy’s Focus Series Interview, “Data-Driven Mailing Helps Heat Up Untapped Seattle Market.”
Tips for Success
In recent years, hundreds of communities have been working to promote home energy upgrades through programs such as the Better Buildings Neighborhood Program, Home Performance with ENERGY STAR, utility-sponsored programs, and others. The following tips present the top lessons these programs want to share related to this handbook. This list is not exhaustive.
Many programs found that market research can help identify, segment, and characterize audiences to understand how to prioritize them. A comprehensive evaluation of over 140 programs across the United States found that programs had greater success when they identified specific target populations within their larger target area, then tailored their outreach to the size of the target populations. Consider prioritizing audiences based on parameters such as demographics, values, housing type, fuel source, potential for savings, common problems with homes, property ownership structure, or program entry point (e.g., remodeling opportunities). For a starting point in your targeting efforts, look online for existing market segmentation data (e.g., municipal records, Zillow, a Nielsen segmentation system called PRIZM, U.S. Census Bureau).
In Their Own Words: Benefits of Market Segmentation
Source: U.S. Department of Energy, 2012.
- The ShopSmart with JEA program in Jacksonville, Florida, purchased Neilson PRIZM lifestyle segmentation data, which included demographic, consumer behavior, and geographic information, to identify, understand, and reach potential home energy upgrade customers. The data divided consumers into 66 demographically and behaviorally distinct segments. Through this market analysis, ShopSmart with JEA discovered that out of those segments, one of the most promising demographics to market its program offerings was older people without children. ShopSmart was able to use this information to market the program specifically to this demographic, as well as identify and target new demographics that had not been active in the program previously.
- In Seattle, Washington, owners of oil-heated homes are ineligible for city-sponsored electric and gas utility rebates. Community Power Works purchased a mailing list from Data Marketing, Inc., that identified all owners of oil-heated homes in the city so the program could reach this previously untapped market. Given the lower efficiency and high cost of heating oil, the program recognized the energy and cost savings potential for these Seattle homeowners and engaged them in undertaking home energy upgrades by focusing outreach on the potential dollar savings that could be achieved by replacing old oil heaters. More than 700 Community Power Works customers who received the mailing then signed up for upgrades between April and August 2012. In the 11 months prior to the first mailing, only 20% of Community Power Works' upgrade projects involved oil-heated homes, and during the six months following the mailing, 50% of the homes were oil-heated. Among those homes, nearly 75% switched from oil heating to high-efficiency electric heating or high-efficiency electric heat pumps, as of mid-December 2012.
- California utilities provided several examples of market segmentation that targeted energy efficiency programs. The report “Market Segmentation and Energy Efficiency Program Design” by the California Institute for Energy and Environment (CIEE) provides an overview of market segmentation purposes, examples, and methodologies. Another CIEE report, “Behavior Assumptions Underlying California Residential Sector Energy Efficiency Programs,” examines how to influence customer behavior and choice.
Target program messages to what customers want, not what the program does. Although residential energy efficiency programs deliver energy efficiency services, customers are more likely to respond to offers of comfort, cost savings, increased home value, health, community pride, or something else they need and value.
- The RePower program in Washington state customized its marketing and outreach strategies to reach the environmentally conscious residents of Bainbridge Island, Washington, and Bremerton, Washington, a neighboring community with a lower income demographic. In Bainbridge, messaging focused on environmental stewardship, and an Island Energy Dashboard displayed real-time energy use in public spaces, such as local businesses and commuter ferries. Messaging geared toward Bremerton residents, meanwhile, emphasized job creation and reduced utility bills. Each location had its own community-specific website, color scheme, print advertising, online promotions, and case studies highlighting local energy champions to drive demand for residential energy upgrades.
In Their Own Words: Messaging to Motivate
Source: U.S. Department of Energy, 2012.
- In Florida, solar energy is in high demand, so the Solar and Energy Loan Fund (SELF) used this as a gateway to reach homeowners. Through SELF, homeowners could receive a loan for solar energy upgrades after meeting certain energy efficiency thresholds. For example, if a home energy assessment showed that the home's envelope was already sealed (or would be sealed as part of the work), a homeowner could qualify for a loan for solar panels offered by the program.
Following up on leads that result from your marketing and outreach efforts quickly and consistently will help your program convert them from interested parties to satisfied customers. Many programs found a sizable drop-off in action if they or their contractors were not able to follow up within a few days to a week. Additionally, contacting initially interested participants regularly (e.g., through a monthly email, with a phone call every few months) was also a successful strategy for turning potential customers into paying customers.
One successful approach programs used to maintain this connection was through energy advisors who followed up with homeowners after their initial interest and coached them through the home energy upgrade process.
- The Greater Cincinnati Energy Alliance (GCEA) realized that many of the homeowners who signed up to learn more about the program were getting stuck between creating their online profile to initiate program participation and completing a home energy assessment, and then between the energy assessment and home energy upgrade stages. Instead of accepting these customers as losses, GCEA had its full-time energy advisor make phone calls to each of these customers to learn why they were not completing their home energy assessments or upgrades and to explain to customers how to move on to the next stage. By explaining the entire upgrade process to individuals, GCEA was able to ensure that potential customers did not drop out of the program simply because their questions were not answered, they did not understand how the program worked, or they forgot that they signed up in the first place. Of the customers the energy advisor contacted, 50% who completed assessments followed through to complete home energy upgrades.
- The Denver Energy Challenge provided customers with free energy advisors who recommended energy improvements and guided participants through the process. The program found that those advisors with a background in customer service had a better conversion rate than those whose expertise focused on building science. Overall, three out of every four customers who worked with an energy advisor went on to complete a home energy upgrade. Although not all participants made all of the improvements recommended at once, because Denver’s energy advisors kept in touch with participants, program staff reported that many homeowners completed additional upgrades later on in the process.
- Connecticut’s Neighbor to Neighbor Energy Challenge used energy advisors and a customer management database to help ensure that their contractors were following up on leads and following through with customers in a timely manner. Through weekly meetings with contractors and daily monitoring of contractor activities, the program’s energy advisors made sure contractors were leading potential customers through the program’s next steps. The program set up its database to send automatic reminders to contractors when they needed to take a new step with the homeowner. This regular follow-up helped ensure that homeowners heard back from contractors within a certain number of days, depending on where they were in the process.
- Michigan Saves, formerly BetterBuildings for Michigan, saw more homeowners undertake upgrades in cities where it held a neighborhood sweep and followed the sweep with a city-wide offering a year later. While the program initially planned for a timeframe of four to six weeks for homeowners to decide whether or not to undertake a home energy upgrade, following up with customers after giving them time to better understand the program’s offer helped Michigan Saves achieve success. Based on the success of the initial follow-up offers, Michigan Saves lengthened the time for each sweep to a full year. Overall, the program was able to complete nearly 8,000 home energy assessments and more than 6,300 home energy upgrades.
Resources
The following resources provide topical information related to this handbook. The resources include a variety of information ranging from case studies and examples to presentations and webcasts. The U.S. Department of Energy does not endorse these materials.
The presentation covers evaluation results and recommendations for California’s Statewide Marketing, Education and Outreach (SW ME&O) program.
The presentation covers evaluation findings from California’s Statewide Marketing, Education and Outreach (SW ME&O) program.
This presentation shares how the Neighbor to Neighbor Energy Challenge collected and evaluated data and used the results to improve its program.
This summary from a Better Buildings Residential Network peer exchange call focused on how to identify stakeholders and map them to determine the best engagement strategy.
This presentation describes steps programs can take to obtain useful feedback from customers regarding their programs.
This presentation covers the importance of collecting and evaluating program data, including data related to marketing efforts.
This sample email survey template, created by the Better Buildings Neighborhood Program, was designed for programs to develop their own survey of successful program participants in order to assess customer experience.
This sample phone survey template, created by the Better Buildings Neighborhood Program, was designed for programs to use with applicants who have been screened out from participating in a program.
This sample phone survey template for program drop-outs, created by the Better Buildings Neighborhood Program, was designed for programs to find out why applicants that applied to participate in a program ultimately dropped out.
This guide provides an assessment of various approaches to Marketing & Outreach for home energy efficiency improvements.