Tracking marketing and outreach objectives is the first step in assessing and improving your marketing and outreach activities. As part of your program design evaluation plan, your marketing and outreach evaluation plan will specify what metrics need to be tracked to gauge success. Most successful programs review these metrics in a dashboard on a monthly, weekly, or even daily basis.

As discussed in the evaluation plan handbook, some key metrics you might want to monitor include:

• Number of leads, assessments, and upgrades associated with specific marketing campaigns
• Cost per lead achieved through a specific outreach channel
• Conversion of leads to assessments and upgrades
• Percentage of customers taking advantage of incentives based on an outreach campaign
• Effectiveness of sales training or contractor marketing efforts, based on contractor success
• Amount of time it takes for a customer to receive a call back from the program after expressing interest during an outreach event.

Depending on how much data you collect, tracking marketing data may require more than simple spreadsheets, especially if you have multiple marketing strategies. For example:

• The Connecticut Neighbor-to-Neighbor Energy Challenge program used a customer relationship management database to track and compare the effectiveness of different marketing strategies.

Your overall program evaluation might include a more formal evaluation process, including a third-party review, that can provide insights into the effectiveness of your marketing and outreach activities. Coordinate with program staff who are conducting this evaluation to ensure that marketing and outreach-related feedback is solicited and shared.

Depending on the metric you are measuring, you may be able to analyze results immediately (e.g., number of homeowners that sign up for home energy assessments at an event) or not until later (e.g., increase in website visitors due to online advertisements). Be patient, but be prepared to review and analyze your data as it becomes available.

Enhabit Tracks Inquiries to Determine What Works

Promotional codes in your marketing materials that consumers reference when inquiring about the program will help track which materials and tactics are bringing customers to your program. Enhabit, formerly Clean Energy Works Oregon, used coupon codes like the one shown at right on marketing materials and asked homeowners to refer to the codes when they contacted the program. Enhabit also provided contractors and community groups codes to add to outreach materials to track the effectiveness of outreach efforts. Program staff said promotional codes helped make prospective participants feel like they had something special, and that they created a sense of urgency to motivate customers. Enhabit and its participating contractors also used the codes to track the efficacy of outreach materials.

Promotional Flier Created by Enhabit Contractor

Abacus Energy Solutions, LLC, an Enhabit contractor, developed this flier that included a promotional code to be entered on Enhabit’s website.

Source: Enhabit, 2011.

In addition to quantitative metrics, collecting qualitative information from customers and partners, such as feedback on your marketing tactics or outreach materials, can help you assess marketing and outreach efforts and make continuous improvements.

Customer surveys can help you evaluate which of your marketing activities and materials are working best, and which are not as effective. Your survey might focus only on marketing and outreach activities (e.g., a qualitative survey to see which messages, marketing methods, and outreach are working best to motivate customers), or marketing and outreach-related questions can be added to an overall program customer survey. Choose whichever method—or both—work best for your needs and budget.

The Survey Says…

Surveys can be a useful tool for collecting feedback from customers on the effectiveness of your marketing and outreach activities. The tools below provide information on what questions to ask which types of customers, from those who completed upgrades to those who dropped out of your program, so you can collect quality customer feedback that can help improve marketing and outreach efforts.

• The U.S. Department of Energy’s Program Evaluation Topics & Questions Library for Program Participants provides a good starting point for developing a survey that collects qualitative data from program participants. Following are some questions related to marketing and outreach that you might want to ask participating homeowners in a customer survey.
  • How did you first hear of the project?
    • Tracking the origin of customer entry into a program is essential to understanding what approaches are working best, and where expectations might not have been met.
  • What were your motivations for responding to a given outreach approach?
    • This can show which methods and messaging were most compelling. To limit the variety of responses this question could elicit, consider providing a list of different motivations and ask participants to rate them on a scale ranging from not motivating to significantly motivating.
  • Did you have any difficulty accessing the program’s website, application forms, or finding answers to your questions on the program’s website?
    • This question can be expanded to cover specifics but should get at the ease of using program outreach to complete various steps of the process.
  • What factors motivated you to: a) sign up for the program; b) complete a home energy assessment; c) invest in energy upgrades?
    • The answers to questions like this can be multiple choice, or you can ask participants to rate prepared responses on a scale ranging from not important to very important.
• The U.S. Department of Energy’s Example of Survey for Successful Participants provides sample questions that can be emailed to customers before or after they complete the upgrade process. Surveying customers who complete the upgrade process can offer insights into your marketing materials’ strengths, weaknesses, and room for improvement. Marketing and outreach-related questions you might ask in an email survey to customers who completed upgrades include:
  • Which materials and activities motivated you to take action?
  • Was the website helpful at explaining program offerings? Explaining the upgrade process? Inspiring action?
• The U.S. Department of Energy’s Example Phone Survey for Screened-Out Applicants includes sample questions to ask customers who initiated program participation but dropped out at some point. Customer feedback to these questions can reveal process or messaging improvements that might need to be made to your marketing and outreach. Questions you might ask in a phone survey to customers that dropped out of the program include:
  • Did your experience interacting with the program’s systems, staff, or partners dissuade you from continuing?
  • Was the program offer not enticing enough or not clearly presented?
  • Was there a difference between how the program was marketed to you, compared to how you perceived the program after receiving more details?

 

See the Toolbox in this handbook for more survey examples. 

Beyond solicited input, customer feedback may come to your program unsolicited, through emails, phone calls, or letters. A tracking mechanism that allows for input from a variety of sources, not just those you plan to collect feedback from, will make it easier to evaluate marketing and outreach activities.

Marketing and outreach partners should also be a source of feedback.

• Regular check-ins with partners can provide the opportunity for your program to collect this feedback.
• Contractor partners typically have the most face-to-face contact with homeowners, so they might be able to report back on what they hear from customers.
• Call center staff can solicit feedback as well since they are speaking directly with customers.

Cincinnati Assessment and Improvements Lead to Significant Increase in Home Energy Upgrades

Like many programs, the Greater Cincinnati Energy Alliance (GCEA) had a pool of homeowners who signed up to learn more about the program or to receive a home energy assessment. GCEA realized, however, that many of the customers did not continue beyond creating their online profile to complete their home energy upgrade. Instead of accepting these customers as losses, GCEA had its full-time energy advisor make phone calls to each of these customers to learn why they were not completing their assessments or upgrades. Of the customers the energy advisor contacted, 50% who completed assessments followed through to complete home energy upgrades. After implementing this energy advisor model, GCEA experienced a 40% conversion rate from home energy assessments to upgrades.

Once you have collected feedback and tracked metrics identified in your evaluation plan, the next step is to review the data and customer feedback collected to understand which activities and materials are working, and which are not. Take the time to review the data and feedback received against these indicators of success.

When evaluating customer feedback, look at how customers and potential customers reacted to your program’s marketing materials and offerings in the context of your marketing and outreach objectives. Your staff and partners might love a marketing tactic that was used, but if it did not motivate customers to take actions outlined in your objectives, it did not achieve success. Evaluating data is crucial to helping you assess what is working and what is not and enables you to make informed adjustments as needed.

You might want to schedule the review and assessment of marketing and outreach data to coincide with review and assessment of data from other program components (e.g., Program Design & Customer Experience, Financing, and Contractor Engagement & Workforce Development) so that you can make the most comprehensive assessment of successful approaches and identify options for improvement where needed.

Two Bills, One Problem

Energy Impact Illinois created an award-winning video series, entitled “A Tale of Two Bills.” Unfortunately, the video series did not bring in any customers. Consequently, Energy Impact Illinois shifted their marketing focus to a more grassroots campaign.

Shifting to the grassroots approach, the “Two Bills” were still used to help establish the program’s brand among a broad audience, but new videos included a clear call to action. By combining a grassroots approach with broader marketing efforts that included the “Two Bills,” the program achieved more than 3,700 single-family home energy upgrades.

Source: Energy Impact Illinois, 2011.

Be prepared to review and evaluate your data, and make marketing and outreach adjustments as needed on a regular basis to ensure continuous improvement of your marketing and outreach efforts. Many successful residential energy efficiency programs achieve success by making marketing adjustments mid-stream. Initial assumptions and assessments about target audiences’ interests, needs, and responses do not always hold true, even when research helps inform the process. Market environments also evolve over time. Do not be afraid to reconsider overall marketing strategies, individual tactics, or materials that are not helping to achieve your marketing and outreach objectives.

Mid-Stream Adjustments Give Chicago a Boost

Energy Impact Illinois (EI2) methodically planned out each step of its residential energy efficiency program. Staff spent months conducting thorough market segmentation research, building a polished website, developing catchy advertising, and creating what they thought would be a beneficial loan product. What they did not do was talk directly to their customers. As a result, their marketing and outreach effort did not have a very successful start.

“The market segments, website, advertising, and loan product would have been more effective, or perhaps unnecessary, once we better understood our market through on-the-ground outreach,” shared David Haeg with CNT Energy, which served as the implementation agency for Energy Impact Illinois. “Plus, if we built a small loyal core of happy customers immediately, we would have had more compelling stories to tell the media and future potential customers.”

Mr. Haeg added that programs should attempt to follow a methodical approach to marketing, but cautioned that programs should not wait too long before testing their approach. “[Waiting] can easily result in teams planning and developing materials for far too long and consequently waiting too long to get into the field to see what actually works.” 

Deciding to use face-to-face contact to build neighborhood trust, EI2 focused on door-to-door canvassing as an outreach strategy to encourage consumers to sign up for energy assessments. This strategy, however, neither targeted specific types of homeowners nor garnered serious interest in home energy improvements. Canvass volunteers found those who actually answered the door were more interested in the free giveaways they were offering.

House Party Model

Source: Energy Impact Illinois, 2013.

The lack of sign-ups for energy assessments caused EI2 to again revamp its outreach strategy, opting for a house party model. Trained field staff and contractors conducted an abbreviated energy assessment of a home with the homeowner and a group of their friends and neighbors. Over the course of a year, the program organized 652 house parties, and 3,110 people attended. More than 2,500 of these attendees signed up for assessments on the spot or called EI2’s call center to host a house party—a conversion rate of 82% from house party attendance to assessment sign-up. Read more about EI2’s mid-stream marketing adjustment in the U.S. Department of Energy’s Focus Series Interview, “Energy Impact Illinois Learns That Parties Sell Upgrades”.

Source: Energy Impact Illinois Learns That Parties Sell Upgrades, U.S. Department of Energy, 2013.

The data you gather on how your marketing activities and materials are being received among target audiences will be critical to both internal decision-making and partner contributions. Communicate your marketing data where appropriate and your decisions made about mid-stream improvements or continuous improvement, both internally and to partners.

You can communicate marketing and outreach decisions in reports, memos, presentations, email, face-to-face meetings, or conference calls with your staff and partners. If you already have a regular check-in method established with your partners, use it to update them on marketing and outreach decisions. In your updates, communicate the data and feedback from partners and other information that influenced your decisions, as well as planned mid-stream improvements.

Community Power Works Heats Up Demand With Targeted Marketing

Community Power Works (CPW) of Seattle, Washington, decided to focus on owners of oil-heated homes as part of its spring marketing strategy. Customer feedback from the program’s other campaigns helped the program determine that direct mail letters were the best way to communicate with this audience. The first mailing was successful at getting oil-heated customers to sign up.

The first letter was sent in April 2012. More than 700 CPW customers who signed up for upgrades from April to August reported mailings as the way they heard about the program. Additionally, in the 11 months prior to the first mailing, only 20% of CPW’s upgrade projects involved oil-heated homes. However, during the six months following the mailing, 50% of the homes that started the upgrade process were oil-heated. Among those homes, nearly 75% switched from oil heating to high-efficiency electric heating or high-efficiency electric heat pumps by mid-December 2012.

Not quite ready for the surge in demand that resulted from the mailing, CPW did not have enough qualified contractors at first to deal with the increased heating, ventilation, and air conditioning (HVAC) work. CPW quickly worked to increase their contractor base and engaged HVAC contractors in Seattle known to turn around projects very quickly.

CPW learned that program staff needed to inform contractors that an influx of customers with outdated oil heaters could be expected soon after each mailing, or risk not having the capacity to respond to homeowners who were ready to switch to more efficient models. CPW staff alerted contractors before subsequent mailings were sent to oil-heated homes and when contractors could expect evaluation sign-ups to spike. They also communicated the program’s expectations for turnaround time so contractors could clear schedules to receive the program’s leads.

For more information, read the U.S. Department of Energy’s Focus Series Interview, “Data-Driven Mailing Helps Heat Up Untapped Seattle Market.”