This page introduces users to the Governance (Feedback) tab of Eightfold (EF) Talent Acquisition (TA) Feature Usage & Outcomes Dashboard. The target audience is users of the dashboard, including the Head of Talent Acquisition, Talent Ops, Eightfold Admins, Recruiting Managers, Recruiters, and Sourcers.

Eightfold’s Feedback features covered

Following features are covered under Governance (Feedback) tab: Quick & Interview Feedback

Filter controls

Dashboard filter controls can be used to refine or subset the data shown in the dashboard. For example, to select a certain date range or select a particular feature.

Filter controls are located at the top of the dashboard. Filter controls have to be read with context based on the type of tab.

Sections

• Overview: This section highlights the top level adoption and outcome metrics associated with feedbacks given for interviews
• User Activity: This section highlights the average weekly and monthly active users of feedback feature, their respective retention percent and how that trends with time
• Feedback Efficiency: This section highlights the efficiency metrics associated with feedbacks such as feedback response time, response rate, pending feedbacks, % positive feedbacks etc. breakdown by feedback requestor
• Scorecards: This section shows feedback stats associated with feedback requestor and responder
• Raw data: This contains all the information at a feedback reviewer and requestor level

Metric Definitions

Overview

Feedback Requesters

• Description: refers to the unique feedback requestors who requested for a feedback in the selected time range
• Dataset(s): ta_feature_feedback_efficiency_and_outcomes

Feedback Responders

• Description: refers to the unique feedback responders who respond to the feedbacks requested in the selected time range
• Dataset(s): ta_feature_feedback_efficiency_and_outcomes

Feedbacks Requested

• Description: This refers to the unique number of feedbacks requested in the selected time range.
• Dataset(s): ta_feature_feedback_efficiency_and_outcomes

Response Rate

• Description: The ratio of total feedbacks submitted (for the feedbacks requested in the selected time frame) and the total number of feedbacks requested
• Formula in Quicksight calculation:
  • sum({feedbacks_submitted})/ sum({feedbacks_requested})

• Dataset(s): tafeaturefeedbackefficiencyand_outcomes

% Positive Feedbacks

• Description: The percentage of positive feedbacks submitted among all the feedbacks submitted for the feedbacks requested in the selected time range
• Formula in Quicksight calculation:
  • {positive_feedbacks}/{feedbacks_submitted}

• Dataset(s): tafeaturefeedbackefficiencyand_outcomes

Average time to respond

• Description: The time taken to respond to respond to the feedbacks submitted for the feedbacks requested in the selected time range
• Dataset(s): ta_feature_feedback_efficiency_and_outcomes

Feedback forms used per position

• Description: The unique number of feedbacks forms per position used in the selected time range
• Formula in Quicksight calculation:
  • sum({feedback_forms_used})/ distinct_count({position_id})

• Dataset(s): tafeaturefeedbackefficiencyand_outcomes

Recruiters Usage till date

• Description: this visual gives info about the total employees with a recruiter role and among them how many (# & %) are requesting for feedback
  • Example: The image below indicates that there are totally 52 employees with a recruiter role and among them 30 of them have requested for at least 1 feedback till date
  • Note: Only filters applicable to this visual is the ‘Feedback Type’ and it always shows the info till date

• Dataset(s): tafeaturefeedbackbreadthofusageacross_users

Usage across Filled Positions till date

• Description: this visual gives info about the total positions with at least 1 hire and number of positions (# & %) which have used feedback forms for interviewing till date
  • Note: Only filters applicable to this visual is the ‘Feedback Type’ and it always shows the info till date
  • Dataset(s): ta_feature_feedback_breadth_of_usage_across_positions

  Usage across Open Positions till date

  • Description: this visual gives info about the total open positions (positions with status = ‘Open’) as of today, and among them the # and % of open positions using the feedback feature
    • Note: Only filters applicable to this visual is the ‘Feedback Type’ and it always shows the info till date

  • Dataset(s): tafeaturefeedbackbreadthofusageacross_positions

  Efficiency

  Feedbacks Requested, Response Rate, Days to Respond, Feedback Forms used per position

  • Description: same as the KPIs defined above

  % Reviewers who took > 4 days to respond

  • Description: The percentage of reviewers who took more than 4 days to respond to reviews among all the reviewers who responded to the feedbacks submitted during the selected time range
  • Formula in Quicksight calculation:
    • distinct_countIf({reviewer_email}, {reviewers_who_respond_after_4_days} = 1)/distinct_countIf({reviewer_email}, {feedbacks_submitted} >0)

  • Dataset(s): tafeaturefeedbackbreadthofusageacross_positions

  % Feedbacks with follow up enabled

  • Description: The percentage of feedback forms requested with follow up enabled
  • Formula in Quicksight calculation:
    • sum({feedbacks_with_follow_up_enabled})/sum({feedbacks_requested})

  • Dataset(s): tafeaturefeedbackbreadthofusageacross_positions

  Workload

  Feedbacks Requested per Requester per Month

  • Description: This section shows the workload in terms of feedbacks requested per reviewer per month
  • Formula in Quicksight calculation:
    • feedbacks requested per requester per month = sum({feedbacks_requested})/distinct_count({requester_email})/distinct_count( concat(toString(truncDate('YYYY' ,{created_at})),toString(truncDate('MM' ,{created_at}))))

  • Dataset(s): tafeaturefeedbackbreadthofusageacross_positions

  Feedbacks Submitted per Reviewer per Month

  • Description: This section shows the workload in terms of feedbacks submitted per reviewer per month in the selected feedbacks requested time frame
  • Formula in Quicksight calculation:
  • feedbacks submitted per reviewer per month = sum({feedbacks_submitted})/distinct_countIf({requester_email}, {feedbacks_submitted}>0)/distinct_count(concat(toString(truncDate('YYYY' ,{created_at})),toString(truncDate('MM' ,{created_at}))))
  • Dataset(s): ta_feature_feedback_breadth_of_usage_across_positions

  Optimal day to request for feedback

  • Description: This gives tips for the feedback requestor on which day of the week to request for feedback in order to get the quickest response
    • Dataset(s): ta_feature_feedback_breadth_of_usage_across_positions

  Insights based on seniority level of reviewer

  • Description: This gives insight into which level of seniority is taking the minimum and maximum amount of time to respond to feedback (Note: The seniority level is inferred by Eightfold based on the title of the role)
  • Dataset(s): ta_feature_feedback_breadth_of_usage_across_positions

  Location with maximum % reviewers responding after 4 days

  • Description: This gives insight into position location which has the maximum % of reviewers responding after 4 days
  • Dataset(s): ta_feature_feedback_breadth_of_usage_across_positions

  User Activity

  Monthly Adoption

  • Avg monthly Active users: It is the average number of monthly active users using feedback feature in the selected period of time
    • YoY metric: denotes how the Avg monthly Active Users has changed compared to the previous year.

  • Avg monthly Retention (Repeat Users) %: It denotes the percentage of users in a given month who were active in the previous month as well

    • YoY metric: denotes how the Avg monthly Repeat users % has changed compared to the previous year.

  • Dataset(s): tafeatureengagementadoptionby_month

  Feedback Requesters

  • Description: Same as previous.
  • Formula in Quicksight calculation:
    • Feedback requestors breakdown by Feedback type
    • Feedbacks requested breakdown by Feedback requestor

  • Dataset(s): tafeaturefeedbackefficiencyand_outcomes

  Feedback Responders

  • Description: Same as previous.
  • Formula in Quicksight calculation:
    • Feedback responder count breakdown by Feedback type
    • Feedbacks submitted breakdown by Feedback reviewer

  • Dataset(s): tafeaturefeedbackefficiencyand_outcomes

  Feedback Efficiency

  This section contains the breakdown of the metrics that we displayed in the Overview section.

  • KPI (on the left): denotes the aggregate value of the metric for a selected period of time
  • Trended chart (in the middle): displays the KPI breakdown by feature and how it trends with time. The X-axis always denotes the timestamp associated with the feedback requested timestamp
  • User level breakdown (on the right): displays the KPI breakdown by user and how it trends with time. The X-axis always denotes the timestamp associated with the feedback requested timestamp

  Note: Metrics which are not part of the Overview section are covered below

  Feedbacks pending

  • Description: This refers to the number of unique feedback requests which are are yet to get a response from the feedback reviewer
  • Formula in Quicksight calculation:
    • Pending feedbacks across feedback type by feedback requested month
      • Pending feedbacks by reviewer by feedback requested month
        • Dataset(s): ta_feature_feedback_efficiency_and_outcomes

  Avg Days to Fill across Time taken to Respond categories (Interview Feedback only considered) (graph)

  • Description: This graph gives insight into how the average days to fill varies with variation in average days taken to respond to ‘Interview’ feedback
    • How to read the data?
      • In the graph above, let’s consider the positions for which it takes 7 days on an average for reviewers to respond. In these cases, it takes on an avg 125 days for a position to get filled
        • Formula in Quicksight calculation:

          • Time to Respond Categories = ifelse(
            {time_to_respond_in_days} < 1, '<1 day',
            {time_to_respond_in_days} >= 1 AND {time_to_respond_in_days} < 3 , '1-2 days',
            {time_to_respond_in_days} >= 3 AND {time_to_respond_in_days} < 5 , '3-4 days',
            {time_to_respond_in_days} >= 5 AND {time_to_respond_in_days} < 7 , '5-6 days',
            {time_to_respond_in_days} >= 7, '7 days',
            'Others' )
            

            • Dataset(s): ta_feature_feedback_efficiency_and_outcomes

  Scorecards

  Feedback Requester Score card

  • Description: This table gives an overall summary of the feedbacks requested per requestor, response rate, % feedbacks with follow up enabled as well as the Unique Positions for which they have requested for feedback
  • Formula in Quicksight calculation:

• Dataset(s): tafeaturefeedbackefficiencyand_outcomes

Reviewer Score card

• Description: This table gives an overall summary of the feedbacks requested per reviewer, their response rate, pending feedbacks, Average time to respond, % Feedbacks with response after 4 days, % Positive Feedbacks

• Formula in Quicksight calculation:

• Dataset(s): tafeaturefeedbackefficiencyand_outcomes

Datasets Used

Ta_feature_feedback_breadth_of_usage_across_users

SELECT 
count(distinct t1.employee_email) as total_employees,
count(distinct case when requester_email IS NOT NULL then t1.employee_email end) as total_active_employees_requesting_feedback,

count(distinct case when is_recruiter then t1.employee_email end) as total_recruiters,
count(distinct case when is_recruiter AND (t2.created_at IS NOT NULL)  then t1.employee_email end) as total_active_recruiters_requesting_feeback,

count(distinct case when feedback_type = 'Quick Feedback'  and (t2.created_at IS NOT NULL)  then t1.employee_email end) as total_active_employees_requesting_quick_feeback,
count(distinct case when feedback_type = 'Interview Feedback'  and (t2.created_at IS NOT NULL)  then t1.employee_email end) as total_active_employees_requesting_interview_feeback,

count(distinct case when is_recruiter AND feedback_type = 'Quick Feedback'  and (t2.created_at IS NOT NULL)  then t1.employee_email end) as total_active_recruiters_requesting_quick_feeback,
count(distinct case when is_recruiter AND feedback_type = 'Interview Feedback'  and (t2.created_at IS NOT NULL)  then t1.employee_email end) as total_active_recruiters_requesting_interview_feeback

FROM 
(
          select employee_email, profile_id, hiring_date, is_recruiter
          from volkscience_analytics.volkscience_employee 
          WHERE is_alumni = FALSE
          and termination_date IS NULL
)t1
LEFT JOIN
(
SELECT 
case when feedback_type = 'quick_feedback' then 'Quick Feedback' 
when feedback_type = 'interview_feedback' then 'Interview Feedback' 
else feedback_type end as feedback_type,
requester_email,
created_at
FROM volkscience_analytics.volkscience_profile_feedback
)t2
on trim(lower(t1.employee_email)) = trim(lower(t2.requester_email))
and t1.hiring_date <= t2.created_at

Ta_feature_feedback_breadth_of_usage_across_positions

SELECT 

count(distinct t1.employee_email) as total_recruiters,
count(distinct case when (t2.created_at IS NOT NULL)  then t1.employee_email end) as total_active_recruiters_using_feeback,
count(distinct case when feedback_type = 'Quick Feedback'  and (t2.created_at IS NOT NULL)  then t1.employee_email end) as total_active_recruiters_using_quick_feeback,
count(distinct case when feedback_type = 'Interview Feedback'  and (t2.created_at IS NOT NULL)  then t1.employee_email end) as total_active_recruiters_using_interview_feeback

FROM 
(
          select employee_email, profile_id, hiring_date, is_recruiter
          from volkscience_analytics.volkscience_employee 
          WHERE is_alumni = FALSE
          and termination_date IS NULL
          and is_recruiter
)t1
LEFT JOIN
(
SELECT 
case when feedback_type = 'quick_feedback' then 'Quick Feedback' 
when feedback_type = 'interview_feedback' then 'Interview Feedback' 
else feedback_type end as feedback_type,
requester_email,
created_at
FROM volkscience_analytics.volkscience_profile_feedback
)t2
on t1.employee_email = t2.requester_email
and t1.hiring_date <= t2.created_at

Ta_feature_feedback_efficiency_and_outcomes

Ta_feature_feedback_adoption_by_week

With feedback_activity_log AS
(SELECT 
datepart('year',created_at) as year,
datepart('week',created_at) as week,
DATE_TRUNC('week' ,created_at) as start_of_week,
case when feedback_type = 'quick_feedback' then 'Quick Feedback' 
when feedback_type = 'interview_feedback' then 'Interview Feedback' 
else feedback_type end as feedback_type,
requester_email as active_user_id,
id,
created_at as activity_timestamp,
LAG(created_at) OVER (PARTITION BY active_user_id ORDER BY created_at) AS previous_activity_timestamp_ts
FROM 
volkscience_analytics.volkscience_profile_feedback
),


weekly_activity_by_feedback_type AS
(
select
year,
week,
start_of_week,
feedback_type,
count(distinct active_user_id) as active_users_this_week,
count(distinct id) as feature_usage,
count(distinct id) /count(distinct active_user_id) as frequency_of_usage_per_user,
count(DISTINCT CASE WHEN DATEDIFF(week, previous_activity_timestamp_ts, activity_timestamp) = 1 THEN active_user_id END) as repeat_users_following_week,
count(DISTINCT CASE WHEN DATEDIFF(week, previous_activity_timestamp_ts, activity_timestamp) = 1 THEN active_user_id END)*1.00 /count(distinct active_user_id) as repeat_users_percent
FROM 
feedback_activity_log
group by 1,2,3, 4
),

aggregate_feedback_weekly_activity AS
(
select
year,
week,
start_of_week,
'All feedbacks' as feedback_type,
count(distinct active_user_id) as active_users_this_week,
count(distinct id) as feature_usage,
count(distinct id) /count(distinct active_user_id) as frequency_of_usage_per_user,
count(DISTINCT CASE WHEN DATEDIFF(week, previous_activity_timestamp_ts, activity_timestamp) = 1 THEN active_user_id END) as repeat_users_following_week,
count(DISTINCT CASE WHEN DATEDIFF(week, previous_activity_timestamp_ts, activity_timestamp) = 1 THEN active_user_id END)*1.00 /count(distinct active_user_id) as repeat_users_percent
FROM 
feedback_activity_log
group by 1,2,3, 4
)

select year, week, start_of_week, feedback_type, active_users_this_week,  feature_usage, frequency_of_usage_per_user, repeat_users_following_week,  repeat_users_percent from weekly_activity_by_feedback_type
UNION 
select year, week, start_of_week, feedback_type, active_users_this_week,  feature_usage, frequency_of_usage_per_user, repeat_users_following_week,  repeat_users_percent  from aggregate_feedback_weekly_activity

ta_feature_feedback_adoption_by _month

With feedback_activity_log AS
(SELECT 
datepart('year',created_at) as year,
datepart('month',created_at) as month,
DATE_TRUNC('month' ,created_at) as start_of_month,
case when feedback_type = 'quick_feedback' then 'Quick Feedback' 
when feedback_type = 'interview_feedback' then 'Interview Feedback' 
else feedback_type end as feedback_type,
requester_email as active_user_id,
id,
created_at as activity_timestamp,
LAG(created_at) OVER (PARTITION BY active_user_id ORDER BY created_at) AS previous_activity_timestamp_ts
FROM 
volkscience_analytics.volkscience_profile_feedback
),


monthly_activity_by_feedback_type AS
(
select
year,
month,
start_of_month,
feedback_type,
count(distinct active_user_id) as active_users_this_month,
count(distinct id) as feature_usage,
count(distinct id) /count(distinct active_user_id) as frequency_of_usage_per_user,
count(DISTINCT CASE WHEN DATEDIFF(month, previous_activity_timestamp_ts, activity_timestamp) = 1 THEN active_user_id END) as repeat_users_following_month,
count(DISTINCT CASE WHEN DATEDIFF(month, previous_activity_timestamp_ts, activity_timestamp) = 1 THEN active_user_id END)*1.00 /count(distinct active_user_id) as repeat_users_percent
FROM 
feedback_activity_log
group by 1,2,3, 4
),

aggregate_feedback_monthly_activity AS
(
select
year,
month,
start_of_month,
'All feedbacks' as feedback_type,
count(distinct active_user_id) as active_users_this_month,
count(distinct id) as feature_usage,
count(distinct id) /count(distinct active_user_id) as frequency_of_usage_per_user,
count(DISTINCT CASE WHEN DATEDIFF(month, previous_activity_timestamp_ts, activity_timestamp) = 1 THEN active_user_id END) as repeat_users_following_month,
count(DISTINCT CASE WHEN DATEDIFF(month, previous_activity_timestamp_ts, activity_timestamp) = 1 THEN active_user_id END)*1.00 /count(distinct active_user_id) as repeat_users_percent
FROM 
feedback_activity_log
group by 1,2,3, 4
)

select year, month, start_of_month, feedback_type, active_users_this_month,  feature_usage, frequency_of_usage_per_user, repeat_users_following_month,  repeat_users_percent from monthly_activity_by_feedback_type
UNION 
select year, month, start_of_month, feedback_type, active_users_this_month,  feature_usage, frequency_of_usage_per_user, repeat_users_following_month,  repeat_users_percent  from aggregate_feedback_monthly_activity