This page introduces users to the Engagement tab of Eightfold (EF) Talent Acquisition (TA) Feature Usage & Outcomes Dashboard. The target audience is users of the dashboard, including the Head of Talent Acquisition, Talent Ops, Eightfold Admins, Recruiting Managers, Recruiters, and Sourcers.

Eightfold’s Engagement features covered

Following features are covered under Engagement tab: Campaigns, Emails, SMS, Whatsapp

Filter controls

Dashboard filter controls can be used to refine or subset the data shown in the dashboard. For example, to select a certain date range or select a particular feature.

Filter controls are located at the top of the dashboard. Filter controls have to be read with context based on the type of tab.

Sections

• Overview: This section highlights the top level adoption and outcome metrics associated with all the Eightfold Engagement Features
• User Activity: This section highlights the average weekly and monthly active users, their respective retention percent and how that trends with time
• Hiring Outcomes: This section highlights the funnel metrics after the candidate are being contacted
• Hiring Efficiency: This section highlights the time efficiency related metrics such as time taken to respond as well as the days between first contact and application conversion
• Key Insights: This section helps in giving tips to the users who use these engagement features for better response rate and response time
• Wall of Fame: This section highlights the top performing users, Location and Job Functions
• Raw Data: This section has the user and candidate level details of the candidates who have been outreached using Campaigns, Emails, SMS & Whatsapp

Metric Definitions

Overview

Active users

• Description: Users who actively use one of the Engagement features (Campaigns, Emails, SMS, Whatsapp) in the selected period of time
• Formula in Quicksight calculation
  • distinct_count(active_user_id)

• Dataset(s): ta_feature_engagement_efficiency_and_outcomes

Candidates reached out

• Description: Unique candidates who were reached out using EF engagement features in the selected period of time
• Formula in Quicksight calculation:
  • Quicksight formula: distinct_count(profile_id)

• Dataset(s): ta_feature_engagement_efficiency_and_outcomes

Response Rate

• Description: The ratio of the Candidates who responded among the candidates who were contacted in the time period selected
• Formula in Quicksight calculation:
  • Quicksight formula: {Responded Candidates}/{Candidates Outreached}
    • Responded Candidates = distinct_countIf({profile_id},isNotNull({reply_time}))
    • Candidates Outreached = distinct_count({profile_id})
• Dataset(s): ta_feature_engagement_efficiency_and_outcomes

Days taken to Respond

• Description: The avg time taken for the candidate to respond when they are contacted for the first time
• Formula in Quicksight calculation:
  • dateDiff({First Contact Date},{First Reply Date},'DD')
    • {First Contact Date} = minOver({created_at},[{active_user_id},{position_id}, {profile_id}],PRE_AGG)
    • {First Reply Date} = minOver({reply_time},[{active_user_id},{position_id}, {profile_id}],PRE_AGG)
• Dataset(s): ta_feature_engagement_efficiency_and_outcomes

% Applied

• Description: The percentage of users who became applicants among the candidates who were contacted using EF engagement features
• Formula in Quicksight calculation:
  • Applicants/{Candidates Outreached}
    • Applicants = distinct_countIf({profile_id},isNotNull({min_application_ts}))
    • {Candidates Outreached} = distinct_count({profile_id})
• Dataset(s): ta_feature_engagement_efficiency_and_outcomes

% Hired

• Description: The percentage of users who were advanced to the hired stage among the candidates who were contacted using EF engagement features
• Formula in Quicksight calculation:
  • {Hired Candidates}/{Candidates Outreached}
    • Hired Candidates = distinct_countIf({profile_id},isNotNull({min_hired_date}))
    • Candidates Outreached = distinct_count({profile_id})
• Dataset(s): ta_feature_engagement_efficiency_and_outcomes

% Active Employees

• Description: The share of current employees (excluding alumni) using the EF engagement features (this metric will change only with ‘Engagement Feature Type’ filter and will always show you the value till date)
• Formula in Quicksight calculation:
  • Numerator =

    ifelse(${EngagementFeatureType} = 'Campaign', {active_employees_using_campaign},
    ${EngagementFeatureType} = 'Email', {active_employees_using_email},
    ${EngagementFeatureType} = 'SMS', {active_employees_using_sms},
    ${EngagementFeatureType} = 'Whatsapp', {active_employees_using_whatsapp},
    {total_active_employees})
    

  • Denominator = Sum(total_employees)

• Dataset(s): ta_feature_engagement_breadth_of_usage

% Active Recruiters (till date)

• Description: The share of employees with a recruiter role (excluding alumni) using the EF engagement features (this metric will change only with ‘Engagement Feature Type’ filter and will always show you the value till date)
• Formula in Quicksight calculation:
  • Numerator =

    ifelse(${EngagementFeatureType} = 'Campaign', {active_recruiters_using_campaign},
    ${EngagementFeatureType} = 'Email', {active_recruiters_using_email},
    ${EngagementFeatureType} = 'SMS', {active_recruiters_using_sms},
    ${EngagementFeatureType} = 'Whatsapp', {active_recruiters_using_whatsapp},
    {total_active_recuiters})
    

  • Denominator = Sum(total_recruiter)

• Dataset(s): ta_feature_engagement_breadth_of_usage

% Active Hiring Managers (till date)

• Description: The share of employees with a hiring manager role (excluding alumni) using the EF engagement features (this metric will change only with ‘Engagement Feature Type’ filter and will always show you the value till date)
• Formula in Quicksight calculation:
  • Numerator =

    ifelse(${EngagementFeatureType} = 'Campaign', {active_hiring_managers_using_campaign},
    ${EngagementFeatureType} = 'Email', {active_hiring_managers_using_email},
    ${EngagementFeatureType} = 'SMS', {active_hiring_managers_using_sms},
    ${EngagementFeatureType} = 'Whatsapp', {active_hiring_managers_using_whatsapp},
    {total_active_hiring_managers})
    

  • Denominator = Sum(total_hiring_managers)

• Dataset(s): ta_feature_engagement_breadth_of_usage

Efficiency & Outcomes

This section gives you a quick peek into how the candidates reached out are being advanced into the stages of hiring as well as the time taken for candidates to respond, to become applicants and get hired from their first contact time.

Funnel conversion

• Candidates Reached out: Unique candidates who were reached out using EF engagement features in the selected period of time.
• % Responded: The ratio of the Candidates who responded among the candidates who were contacted in the time period selected.
• % Applied: The percentage of users who became applicants among the candidates who were contacted using EF engagement features.
• % Phonescreened: Phone Screened candidates / Candidates Reached Out
  • where, Phone Screened candidates = The candidates who have been advanced to the ‘phonescreen’ stage group.

• % Interviewed = Interviewed candidates / Candidates Sourced
  • where, Interviewed candidates = The candidates who have been advanced to the ‘onsite’ or ‘interview’ stage group.

• % Hired: The percentage of users who were advanced to the hired stage among the candidates who were contacted using EF engagement features.

Funnel conversion trend

• The trend lines parallel to the funnel denote how these metrics vary with time and the x-axis denotes the time at which the candidates were reached out.
  • Example: If the % Applied = 40% in May 2023, it means that among the candidates who were reached out in May 2023, 40% of them have got converted as Applicants.

Time taken Insights

• Days taken to Respond: The avg time taken for the candidate to respond when they are contacted for the first time.
• Days between first contact and application submission: The avg days between the first contact date and the application submission date for those who got converted into applicants
  • Formula = dateDiff({First Contact Date},{min_application_ts},'DD')
    • {First Contact Date} = minOver({created_at},[{active_user_id},{position_id}, {profile_id}],PRE_AGG)

• Days between first contact and hired date: The avg days between the first contact date and the hired date for those who were advanced to the hired stage

  • Formula = dateDiff({First Contact Date},{min_hired_date},'DD')
    • {First Contact Date} = minOver({created_at},[{active_user_id},{position_id}, {profile_id}],PRE_AGG)

Key Insights

This section gives the engagement users a brief idea about how many candidates to contact in order to get x number of applicants or hires

User Activity

Weekly Adoption

• Avg Weekly Active users: It is the average number of Weekly active users using one of these Eightfold engagement Features in the selected time frame
  • YoY metric: denotes the % change in Average Weekly Active Users compared to the previous year

• Avg Weekly Retention (Repeat Users) %: It denotes the percentage of users in a given week who were active in the previous week as well

  • YoY metric: denotes the change in Average Weekly Repeat Users % compared to the previous year

• Dataset(s): ta_feature_engagement_adoption_by_week

Monthly Adoption

• Avg monthly Active users: It is the average number of monthly active users using one of these Eightfold engagement Features in the selected time frame
  • YoY metric: denotes how the Avg monthly Active Users has changed compared to the previous year.

• Avg monthly Retention (Repeat Users) %: It denotes the percentage of users in a given month who were active in the previous month as well

  • YoY metric: denotes how the Avg monthly Repeat users % has changed compared to the previous year.

• Dataset(s): ta_feature_engagement_adoption_by_month

Overall Adoption

• Description: This section gives you the aggregate number of active engagement users for a selected period of time, how they trend with time across the engagement features and user level breakdown
• Dataset(s): ta_feature_engagement_efficiency_and_outcomes

Hiring Outcomes & Hiring Efficiency

This section contains the breakdown of the metrics that we displayed in the Overview section.

• KPI (on the left): denotes the aggregate value of the metric for a selected period of time
• Trended chart (in the middle): displays the KPI breakdown by feature and how it trends with time. The X-axis always denotes the timestamp associated with the engagement activity
• User level breakdown (on the right): displays the KPI breakdown by user and how it trends with time. The X-axis always denotes the timestamp associated with the engagement activity

Key Insights

This section gives insights for active engagement users on when would be the best time to contact and which medium suits them the best based on historical data.

Wall of Fame

This section highlights all the top performing engagement cohorts in terms of:

• candidates outreach
• response rate
• hires conversion
• response time

Raw Data

This section gives you the candidate level information about all the candidates outreached.

Datasets used

Note: In the queries below, replace the keyword ‘volkscience’ with the relevant customer name to replicate the data pull via data hub

ta_feature_engagement_efficiency_and_outcomes

With candidate_engagement_activity_log AS
(
SELECT 
   case when user_campaign_id IS NOT NULL then 'Campaign' 
        when user_campaign_id IS NULL and trim(conversation_type) = 'email' then 'Email'
        when user_campaign_id IS NULL and trim(conversation_type) = 'sms' then 'SMS' 
        when user_campaign_id IS NULL and trim(conversation_type) = 'whatsapp' then 'Whatsapp'
        else conversation_type end as activity_type,
  conversation_type,
   user_id as active_user_id,
   position_id,
   profile_id,
   id, 
   created_at,
   reply_time
FROM 
volkscience_analytics.volkscience_user_messages
WHERE conversation_id IN ('contact','sms_send','sms_send_success','whatsapp_send')
),

ta_outcomes_log AS
(
SELECT 
profile_id,
position_id,
position_business_unit,
position_job_function,
position_location,
position_location_country,
position_recruiter_name,
position_hiring_manager_name,
(case when application_inferred_gender = 'female' then 1 else 0 end) as diverse_candidate_indicator,
min(phonescreen_ts) as min_phonescreened_date,
min(onsite_interview_ts) as min_interviewed_date,
min(offer_ts) as min_offer_date,
min(hired_advanced_ts) as min_hired_date,
max(application_last_stage_ts) as last_updated_stage,
min(application_ts) as min_application_ts,
avg(application_match_score) as application_match_score
FROM
volkscience_analytics.volkscience_ta_leaders_hiring_outcomes
group by 1, 2, 3, 4, 5, 6, 7, 8, 9
)

SELECT
    activity_type,
    conversation_type,
    active_user_id,
    concat(t3.first_name,concat(' ',t3.last_name)) as active_user_name,
    manager_name,
    t1.position_id,
    t1.profile_id,
    id, 
    created_at,
    reply_time,
    min_application_ts,
    diverse_candidate_indicator,
    application_match_score,
    min_phonescreened_date,
    min_interviewed_date,
    min_hired_date,
    last_updated_stage,
    position_business_unit,
    position_job_function,
    position_location,
    position_location_country,
    position_recruiter_name,
    position_hiring_manager_name,
    employee_title,
    employee_level,
    employee_seniority,
    employee_line_of_business,
    employee_business_unit,
    is_alumni,
    employee_location,
    employee_location_country,
    is_recruiter,
    is_hm,
    is_sourcer,
    is_event_team_external,
    race,
    gender,
    ef_gender,
    ef_race

FROM candidate_engagement_activity_log t1

left join ta_outcomes_log t2
on t1.position_id = t2.position_id
and t1.profile_id = t2.profile_id
and t1.created_at <= t2.min_application_ts
left join 
(select 
employee_email, 
manager_email, 
first_name, 
last_name, 
manager_name,
case when title IS NULL then 'Title not assigned' else title end as employee_title,
case when level IS NULL then 'Level not assigned' else level end as employee_level,
case when seniority IS NULL then 'Seniority not assigned' else seniority end as employee_seniority,
case when lob IS NULL then 'Level of Business not assigned' else lob end as employee_line_of_business,
case when business_unit IS NULL then 'Business unit not assigned' else business_unit end as employee_business_unit,
is_alumni,
case when location IS NULL then 'Location not found' else location end as employee_location,
case when location_country IS NULL then 'Location Country not found' else location end as employee_location_country,
is_recruiter,
is_hm,
is_sourcer,
is_event_team_external,
race,
gender,
ef_gender,
ef_race

from volkscience_analytics.volkscience_employee) t3
on t1.active_user_id = t3.employee_email

ta_feature_engagement_adoption_by_month

With candidate_engagement_activity_log AS
(
SELECT 
   datepart('year',created_at) as year,
   datepart('month',created_at) as month,
   DATE_TRUNC('month' , created_at) as start_of_month,
    case when user_campaign_id IS NOT NULL then 'Campaign' 
        when user_campaign_id IS NOT NULL and conversation_type = 'email' then 'Email'
        when user_campaign_id IS NOT NULL and conversation_type = 'sms' then 'SMS' 
        when user_campaign_id IS NOT NULL and conversation_type = 'whatsapp' then 'Whatsapp'
        else conversation_type end as activity_type,
   user_id as active_user_id,
   id,
   created_at as activity_timestamp,
   LAG(created_at) OVER (PARTITION BY active_user_id ORDER BY created_at) AS previous_activity_timestamp_ts
FROM 
volkscience_analytics.volkscience_user_messages
WHERE conversation_id IN ('contact','sms_send','sms_send_success','whatsapp_send')
),

monthly_activity_by_engagement_type AS (
select
t1.year,
t1.month,
t1.start_of_month,
t1.activity_type,
count(distinct t1.active_user_id) as active_users_this_month,
count(distinct id) as messages_sent_count,
count(distinct id) /count(distinct t1.active_user_id) as frequency_of_usage_per_user,
count(DISTINCT CASE WHEN DATEDIFF(month, previous_activity_timestamp_ts, activity_timestamp) = 1 THEN active_user_id END) as repeat_users_following_month,
count(DISTINCT CASE WHEN DATEDIFF(month, previous_activity_timestamp_ts, activity_timestamp) = 1 THEN active_user_id END)*1.00 /count(distinct active_user_id) as repeat_users_percent

FROM
candidate_engagement_activity_log t1

where t1.activity_type != 'others'
group by 1,2,3,4
),

aggregate_engagement_monthly_activity AS (
select
t1.year,
t1.month,
t1.start_of_month,
'All Engagement Activity' as activity_type,
count(distinct t1.active_user_id) as active_users_this_month,
count(distinct id) as messages_sent_count,
count(distinct id) /count(distinct t1.active_user_id) as frequency_of_usage_per_user,
count(DISTINCT CASE WHEN DATEDIFF(month, previous_activity_timestamp_ts, activity_timestamp) = 1 THEN active_user_id END) as repeat_users_following_month,
count(DISTINCT CASE WHEN DATEDIFF(month, previous_activity_timestamp_ts, activity_timestamp) = 1 THEN active_user_id END)*1.00 /count(distinct active_user_id) as repeat_users_percent

FROM
candidate_engagement_activity_log t1

where t1.activity_type != 'others'
group by 1,2,3,4

)

select year, month, start_of_month, activity_type, active_users_this_month,  messages_sent_count, frequency_of_usage_per_user, repeat_users_following_month,  repeat_users_percent from monthly_activity_by_engagement_type
UNION 
select year, month, start_of_month, activity_type, active_users_this_month,  messages_sent_count, frequency_of_usage_per_user, repeat_users_following_month,  repeat_users_percent  from aggregate_engagement_monthly_activity

ta_feature_engagement_adoption_by_week

With candidate_engagement_activity_log AS
(
SELECT 
   datepart('year',created_at) as year,
   datepart('week',created_at) as week,
   DATE_TRUNC('week' , created_at) as start_of_week,
    case when user_campaign_id IS NOT NULL then 'Campaign' 
        when user_campaign_id IS NOT NULL and conversation_type = 'email' then 'Email'
        when user_campaign_id IS NOT NULL and conversation_type = 'sms' then 'SMS' 
        when user_campaign_id IS NOT NULL and conversation_type = 'whatsapp' then 'Whatsapp'
        else conversation_type end as activity_type,
   user_id as active_user_id,
   id,
   created_at as activity_timestamp,
   LAG(created_at) OVER (PARTITION BY active_user_id ORDER BY created_at) AS previous_activity_timestamp_ts
FROM 
volkscience_analytics.volkscience_user_messages
WHERE conversation_id IN ('contact','sms_send','sms_send_success','whatsapp_send')
),

weekly_activity_by_engagement_type AS (
select
t1.year,
t1.week,
t1.start_of_week,
t1.activity_type,
count(distinct t1.active_user_id) as active_users_this_week,
count(distinct id) as messages_sent_count,
count(distinct id) /count(distinct t1.active_user_id) as frequency_of_usage_per_user,
count(DISTINCT CASE WHEN DATEDIFF(week, previous_activity_timestamp_ts, activity_timestamp) = 1 THEN active_user_id END) as repeat_users_following_week,
count(DISTINCT CASE WHEN DATEDIFF(week, previous_activity_timestamp_ts, activity_timestamp) = 1 THEN active_user_id END)*1.00 /count(distinct active_user_id) as repeat_users_percent

FROM
candidate_engagement_activity_log t1

where t1.activity_type != 'others'
group by 1,2,3,4
),

aggregate_engagement_weekly_activity AS (
select
t1.year,
t1.week,
t1.start_of_week,
'All Engagement Activity' as activity_type,
count(distinct t1.active_user_id) as active_users_this_week,
count(distinct id) as messages_sent_count,
count(distinct id) /count(distinct t1.active_user_id) as frequency_of_usage_per_user,
count(DISTINCT CASE WHEN DATEDIFF(week, previous_activity_timestamp_ts, activity_timestamp) = 1 THEN active_user_id END) as repeat_users_following_week,
count(DISTINCT CASE WHEN DATEDIFF(week, previous_activity_timestamp_ts, activity_timestamp) = 1 THEN active_user_id END)*1.00 /count(distinct active_user_id) as repeat_users_percent

FROM
candidate_engagement_activity_log t1

where t1.activity_type != 'others'
group by 1,2,3,4

)

select year, week, start_of_week, activity_type, active_users_this_week,  messages_sent_count, frequency_of_usage_per_user, repeat_users_following_week,  repeat_users_percent from weekly_activity_by_engagement_type
UNION 
select year, week, start_of_week, activity_type, active_users_this_week,  messages_sent_count, frequency_of_usage_per_user, repeat_users_following_week,  repeat_users_percent  from aggregate_engagement_weekly_activity

ta_feature_engagement_breadth_of_usage

With candidate_engagement_activity_log AS
(
SELECT 
   datepart('year',created_at) as year,
   datepart('week',created_at) as week,
   DATE_TRUNC('week' , created_at) as start_of_week,
   case when user_campaign_id IS NOT NULL then 'Campaign' 
        when user_campaign_id IS NULL and trim(conversation_type) = 'email' then 'Email'
        when user_campaign_id IS NULL and trim(conversation_type) = 'sms' then 'SMS' 
        when user_campaign_id IS NULL and trim(conversation_type) = 'whatsapp' then 'Whatsapp'
        else conversation_type end as activity_type,
   user_id as active_user_id,
   id,
   created_at as activity_timestamp,
   LAG(created_at) OVER (PARTITION BY active_user_id ORDER BY created_at) AS previous_activity_timestamp_ts
FROM 
volkscience_analytics.volkscience_user_messages
WHERE conversation_id IN ('contact','sms_send','sms_send_success','whatsapp_send')
),

weekly_activity_by_engagement_type AS (
select
t1.year,
t1.week,
t1.start_of_week,
t1.activity_type,
count(distinct t1.active_user_id) as active_users_this_week,
count(distinct id) as messages_sent_count,
count(distinct id) /count(distinct t1.active_user_id) as frequency_of_usage_per_user,
count(DISTINCT CASE WHEN DATEDIFF(week, previous_activity_timestamp_ts, activity_timestamp) = 1 THEN active_user_id END) as repeat_users_following_week,
count(DISTINCT CASE WHEN DATEDIFF(week, previous_activity_timestamp_ts, activity_timestamp) = 1 THEN active_user_id END)*1.00 /count(distinct active_user_id) as repeat_users_percent

FROM
candidate_engagement_activity_log t1

where t1.activity_type != 'others'
group by 1,2,3,4
),

aggregate_engagement_weekly_activity AS (
select
t1.year,
t1.week,
t1.start_of_week,
'All Engagement Activity' as activity_type,
count(distinct t1.active_user_id) as active_users_this_week,
count(distinct id) as messages_sent_count,
count(distinct id) /count(distinct t1.active_user_id) as frequency_of_usage_per_user,
count(DISTINCT CASE WHEN DATEDIFF(week, previous_activity_timestamp_ts, activity_timestamp) = 1 THEN active_user_id END) as repeat_users_following_week,
count(DISTINCT CASE WHEN DATEDIFF(week, previous_activity_timestamp_ts, activity_timestamp) = 1 THEN active_user_id END)*1.00 /count(distinct active_user_id) as repeat_users_percent

FROM
candidate_engagement_activity_log t1

where t1.activity_type != 'others'
group by 1,2,3,4

)

select year, week, start_of_week, activity_type, active_users_this_week,  messages_sent_count, frequency_of_usage_per_user, repeat_users_following_week,  repeat_users_percent from weekly_activity_by_engagement_type
UNION 
select year, week, start_of_week, activity_type, active_users_this_week,  messages_sent_count, frequency_of_usage_per_user, repeat_users_following_week,  repeat_users_percent  from aggregate_engagement_weekly_activity