TabRanker: Query time logging with label_id and query_id.

When chrome browser is under memory pressure,
TabManager::GetSortedLifecycleUnits is called to return list of tabs
sorted by its reactivation score.

We want to log this event by:
(1) generates (and logs) a query_id for this query.
(2) for each tab, when its reactivation score is calculated, we log its
tab_features (with a random label_id) for this query.
(3) when the tab is later on reactivated or closed, we log the label with
the label_id generated before so that feature-label can be paired.

Please see more details in the bug.

Changes made in this CL:

(1) Extending TabMetricsLogger
chrome/browser/resource_coordinator/tab_metrics_logger.h
chrome/browser/resource_coordinator/tab_metrics_logger.cc
chrome/browser/resource_coordinator/tab_metrics_logger_unittest.cc

Minor change of these functions to also take label_id and query_id as input.
(label_id is passed in as input since it's different per tab, query_id is
saved and set inside TabMetricsLogger since it's the same of all tabs per
discard-query)

(2) Populating the label_id and query_id
chrome/browser/resource_coordinator/tab_activity_watcher.h
chrome/browser/resource_coordinator/tab_activity_watcher.cc
chrome/browser/resource_coordinator/tab_activity_watcher_browsertest.cc

TabActivityWatcher::CalculateReactivationScore and
TabActivityWatcher::WebContentsData::CalculateReactivationScore are extended
to also take also_log_to_ukm as input.

A tab_feature with a random label_id_ is logged whenever
CalculateReactivationScore(true) is called. The same label_id_ is also logged
in LogForegroundedOrClosedMetrics.

We also add logic to block background time feature logging for now.

(3) TabManager changes.
chrome/browser/resource_coordinator/tab_lifecycle_unit.cc
chrome/browser/resource_coordinator/tab_manager.h
chrome/browser/resource_coordinator/tab_manager.cc

These changes are necessary only because
TabManager::GetSortedLifecycleUnits and
TabActivityWatcher::CalculateReactivationScore are called at multiple places.
And we only want to log some of them.

a) TabLifecycleUnitSource::TabLifecycleUnit::GetSortKey is changed back to
   only sort by last_activity_time for now.
b)  is added as private function and
   called inside TabManager::DiscardTabImpl if TabRanker is enabled.
c) CalculateReactivationScore(also_log_to_ukm=true) is called inside
   GetSortedLifecycleUnitsFromTabRanker, which is the only case we log query
   time features.

Bug: 900454

Change-Id: Idc77b0ff6aa3488e10a449dfec6a96d4c61f8da0
Reviewed-on: https://chromium-review.googlesource.com/c/1350445Reviewed-by: Kentaro Hara <haraken@chromium.org>
Reviewed-by: Michael Giuffrida <michaelpg@chromium.org>
Reviewed-by: Robert Kaplow <rkaplow@chromium.org>
Reviewed-by: Sébastien Marchand <sebmarchand@chromium.org>
Commit-Queue: Charles . <charleszhao@chromium.org>
Cr-Commit-Position: refs/heads/master@{#612070}

chrome/browser/resource_coordinator/tab_activity_watcher.cc

@@ -4,8 +4,11 @@
 
 #include "chrome/browser/resource_coordinator/tab_activity_watcher.h"
 
+#include <limits>
+
 #include "base/metrics/histogram_macros.h"
 #include "base/no_destructor.h"
+#include "base/rand_util.h"
 #include "chrome/browser/profiles/profile.h"
 #include "chrome/browser/resource_coordinator/tab_manager_features.h"
 #include "chrome/browser/resource_coordinator/tab_metrics_logger.h"
@@ -53,7 +56,7 @@ class TabActivityWatcher::WebContentsData
   // Calculates the tab reactivation score for a background tab. Returns nullopt
   // if the score could not be calculated, e.g. because the tab is in the
   // foreground.
-  base::Optional<float> CalculateReactivationScore() {
+  base::Optional<float> CalculateReactivationScore(bool also_log_to_ukm) {
     if (web_contents()->IsBeingDestroyed() || backgrounded_time_.is_null())
       return base::nullopt;
 
@@ -62,22 +65,34 @@ class TabActivityWatcher::WebContentsData
     const auto mru = GetMRUFeatures();
     const int lru_index = mru.total - mru.index - 1;
 
-    // If the least recently used index is greater than or equals to N, which
-    // means the tab is not in the oldest N list, we should simply skip it.
-    // The N is defaulted as kMaxInt so that all tabs are scored.
-    if (lru_index >= GetNumOldestTabsToScoreWithTabRanker())
+    // If the least recently used index is greater than both numbers, then we
+    // don't need to score or log the tab; directly return instead.
+    if (lru_index >= GetNumOldestTabsToScoreWithTabRanker() &&
+        lru_index >= GetNumOldestTabsToLogWithTabRanker()) {
       return base::nullopt;
+    }
 
     base::Optional<tab_ranker::TabFeatures> tab = GetTabFeatures(mru);
     if (!tab.has_value())
       return base::nullopt;
 
-    float score;
-    tab_ranker::TabRankerResult result =
-        TabActivityWatcher::GetInstance()->predictor_.ScoreTab(tab.value(),
-                                                               &score);
-    if (result == tab_ranker::TabRankerResult::kSuccess)
-      return score;
+    if (also_log_to_ukm && lru_index < GetNumOldestTabsToLogWithTabRanker()) {
+      // A new label_id_ is generated for this query.
+      // The same label_id_ will be logged with ForegroundedOrClosed event later
+      // on so that TabFeatures can be paired with ForegroundedOrClosed.
+      label_id_ = static_cast<int64_t>(
+          base::RandGenerator(std::numeric_limits<uint64_t>::max() - 1) + 1);
+      TabActivityWatcher::GetInstance()->tab_metrics_logger_->LogTabMetrics(
+          ukm_source_id_, tab.value(), web_contents(), label_id_);
+    }
+
+    if (lru_index < GetNumOldestTabsToScoreWithTabRanker()) {
+      float score;
+      if (TabActivityWatcher::GetInstance()->predictor_.ScoreTab(
+              tab.value(), &score) == tab_ranker::TabRankerResult::kSuccess)
+        return score;
+    }
+
     return base::nullopt;
   }
 
@@ -102,6 +117,9 @@ class TabActivityWatcher::WebContentsData
 
     // Record previous ukm_source_id from the |replaced_tab|.
     previous_ukm_source_id_ = replaced_tab.ukm_source_id_;
+
+    // Copy the replaced label_id_.
+    label_id_ = replaced_tab.label_id_;
   }
 
   // Call when the WebContents is detached from its tab. If the tab is later
@@ -134,11 +152,15 @@ class TabActivityWatcher::WebContentsData
 
   // Logs TabMetrics for the tab if it is considered to be backgrounded.
   void LogTabIfBackgrounded() {
-    if (!backgrounded_time_.is_null()) {
-      base::Optional<tab_ranker::TabFeatures> tab = GetTabFeatures();
-      if (tab.has_value())
-        TabActivityWatcher::GetInstance()->tab_metrics_logger_->LogTabMetrics(
-            ukm_source_id_, tab.value(), web_contents());
+    if (backgrounded_time_.is_null() || DisableBackgroundLogWithTabRanker())
+      return;
+
+    base::Optional<tab_ranker::TabFeatures> tab = GetTabFeatures();
+    if (tab.has_value()) {
+      // Background time logging always logged with label_id == 0, since we
+      // only use label_id for query time logging for now.
+      TabActivityWatcher::GetInstance()->tab_metrics_logger_->LogTabMetrics(
+          ukm_source_id_, tab.value(), web_contents(), 0);
     }
   }
 
@@ -399,6 +421,7 @@ class TabActivityWatcher::WebContentsData
     const auto mru = GetMRUFeatures();
     metrics.mru_index = mru.index;
     metrics.total_tab_count = mru.total;
+    metrics.label_id = label_id_;
 
     const ukm::SourceId source_id = discarded_since_backgrounded_
                                         ? previous_ukm_source_id_
@@ -406,6 +429,14 @@ class TabActivityWatcher::WebContentsData
     TabActivityWatcher::GetInstance()
         ->tab_metrics_logger_->LogForegroundedOrClosedMetrics(source_id,
                                                               metrics);
+    // label_id_ is reset whenever a label is logged.
+    // A new label_id_ is generated when a query happens inside
+    // CalculateReactivationScore, after that this ForegroundedOrClosed logging
+    // can happen many times (tabs may get backgrounded and reactivated several
+    // times). In such cases, we only count the first time as the true label,
+    // the rest are considered to be query time logging irrelevant, for which we
+    // log with label_id == 0.
+    label_id_ = 0;
   }
 
   // Updated when a navigation is finished.
@@ -451,6 +482,9 @@ class TabActivityWatcher::WebContentsData
   // Whether this tab is currently in discarded state.
   bool discarded_since_backgrounded_ = false;
 
+  // An int64 random label to pair TabFeatures with ForegroundedOrClosed event.
+  int64_t label_id_ = 0;
+
   DISALLOW_COPY_AND_ASSIGN(WebContentsData);
 };
 
@@ -467,12 +501,17 @@ TabActivityWatcher::TabActivityWatcher()
 TabActivityWatcher::~TabActivityWatcher() = default;
 
 base::Optional<float> TabActivityWatcher::CalculateReactivationScore(
-    content::WebContents* web_contents) {
+    content::WebContents* web_contents,
+    bool also_log_to_ukm) {
   WebContentsData* web_contents_data =
       WebContentsData::FromWebContents(web_contents);
   if (!web_contents_data)
     return base::nullopt;
-  return web_contents_data->CalculateReactivationScore();
+  return web_contents_data->CalculateReactivationScore(also_log_to_ukm);
+}
+
+void TabActivityWatcher::SetQueryIdForTabMetricsLogger(int64_t query_id) {
+  tab_metrics_logger_->set_query_id(query_id);
 }
 
 void TabActivityWatcher::OnBrowserSetLastActive(Browser* browser) {

chrome/browser/resource_coordinator/tab_activity_watcher.h

@@ -34,7 +34,13 @@ class TabActivityWatcher : public BrowserListObserver,
   // value indicates a higher likelihood of being reactivated.
   // Returns the score if the tab could be scored.
   base::Optional<float> CalculateReactivationScore(
-      content::WebContents* web_contents);
+      content::WebContents* web_contents,
+      bool also_log_to_ukm = false);
+
+  // Generates a new query_id for logging CalculateReactivationScore calls.
+  // This function is called inside TabManager::GetSortedLifecycleUnits when
+  // reactivation scores are queried for all tabs.
+  void SetQueryIdForTabMetricsLogger(int64_t query_id);
 
   // Returns the single instance, creating it if necessary.
   static TabActivityWatcher* GetInstance();

chrome/browser/resource_coordinator/tab_activity_watcher_browsertest.cc

@@ -221,6 +221,17 @@ class TabActivityWatcherUkmTest : public TabActivityWatcherTest {
     return entry->source_id;
   }
 
+  // Gets the latest metric value from the event with given |metric_name|.
+  int64_t GetLatestMetricValue(const std::string& event_name,
+                               const std::string& metric_name) {
+    const std::vector<const ukm::mojom::UkmEntry*> entries =
+        test_ukm_recorder_->GetEntriesByName(event_name);
+    // Check nonemptiness.
+    EXPECT_FALSE(entries.empty());
+
+    return *(test_ukm_recorder_->GetEntryMetric(entries.back(), metric_name));
+  }
+
   std::unique_ptr<UkmEntryChecker> ukm_entry_checker_;
   std::unique_ptr<ukm::TestAutoSetUkmRecorder> test_ukm_recorder_;
 
@@ -482,4 +493,230 @@ IN_PROC_BROWSER_TEST_F(TabActivityWatcherUkmTest,
   }
 }
 
+// Test the query time logging is correct.
+IN_PROC_BROWSER_TEST_F(TabActivityWatcherUkmTest,
+                       LogInCalculateReactivationScore) {
+  // Set Feature params for this test.
+  // (1) background log is disabled, so that only query time logging and
+  // corresponding labels should be logged.
+  // (2) number of oldest tabs to log is set to 2, so that foreground tab should
+  // not be logged.
+  base::test::ScopedFeatureList scoped_feature_list;
+  scoped_feature_list.InitAndEnableFeatureWithParameters(
+      features::kTabRanker,
+      {{"number_of_oldest_tabs_to_log_with_TabRanker", "2"},
+       {"disable_background_log_with_TabRanker", "true"}});
+
+  // Use test clock so tabs have non-zero backgrounded times.
+  base::SimpleTestTickClock test_clock;
+  ScopedSetTickClockForTesting scoped_set_tick_clock_for_testing(&test_clock);
+  test_clock.Advance(base::TimeDelta::FromMinutes(1));
+
+  ui_test_utils::NavigateToURL(browser(), test_urls_[0]);
+  // Insert the tab@1.
+  AddTabAtIndex(1, test_urls_[1], ui::PAGE_TRANSITION_LINK);
+  test_clock.Advance(base::TimeDelta::FromMinutes(1));
+  // Insert the tab@2.
+  AddTabAtIndex(2, test_urls_[2], ui::PAGE_TRANSITION_LINK);
+  test_clock.Advance(base::TimeDelta::FromMinutes(1));
+  // Activate tab@0.
+  browser()->tab_strip_model()->ActivateTabAt(0, kIsUserGesture);
+  test_clock.Advance(base::TimeDelta::FromMinutes(1));
+
+  // Score tab_0 will fail because tab_0 is current foreground tab.
+  base::Optional<float> tab_0 =
+      TabActivityWatcher::GetInstance()->CalculateReactivationScore(
+          browser()->tab_strip_model()->GetWebContentsAt(0), true);
+  EXPECT_FALSE(tab_0.has_value());
+
+  // No metrics should be logged till now.
+  EXPECT_EQ(0, ukm_entry_checker_->NumNewEntriesRecorded(kEntryName));
+  EXPECT_EQ(1, ukm_entry_checker_->NumNewEntriesRecorded(kFOCEntryName));
+  {
+    SCOPED_TRACE("");
+    ukm_entry_checker_->ExpectNewEntry(
+        kFOCEntryName, test_urls_[0],
+        {{TabManager_Background_ForegroundedOrClosed::kIsForegroundedName, 1},
+         {TabManager_Background_ForegroundedOrClosed::kLabelIdName, 0},
+         {TabManager_Background_ForegroundedOrClosed::kIsDiscardedName, 0}});
+  }
+
+  // Set query_id.
+  const int64_t query_id = 1234;
+  TabActivityWatcher::GetInstance()->SetQueryIdForTabMetricsLogger(query_id);
+
+  // tab@1 is scored successfully.
+  base::Optional<float> tab_1 =
+      TabActivityWatcher::GetInstance()->CalculateReactivationScore(
+          browser()->tab_strip_model()->GetWebContentsAt(1), true);
+  EXPECT_TRUE(tab_1.has_value());
+  int64_t label_id_1 = 0;
+  {
+    SCOPED_TRACE("");
+    // tab feature of tab@1 should be logged correctly.
+    UkmMetricMap expected_tab_feature_values = kBasicMetricValues;
+    expected_tab_feature_values[TabManager_TabMetrics::kMRUIndexName] = 2;
+    expected_tab_feature_values
+        [TabManager_TabMetrics::kNumReactivationBeforeName] = 0;
+    expected_tab_feature_values[TabManager_TabMetrics::kTotalTabCountName] = 3;
+    expected_tab_feature_values[TabManager_TabMetrics::kWindowTabCountName] = 3;
+    expected_tab_feature_values[TabManager_TabMetrics::kQueryIdName] = query_id;
+
+    ukm_entry_checker_->ExpectNewEntry(kEntryName, test_urls_[1],
+                                       expected_tab_feature_values);
+    // Get label_id_1 for later verification.
+    label_id_1 =
+        GetLatestMetricValue(kEntryName, TabManager_TabMetrics::kLabelIdName);
+    EXPECT_NE(label_id_1, 0);
+  }
+
+  // Reactivate tab@1 should log a ForegroundedOrClosed event with LabelId as
+  // label_id_1.
+  browser()->tab_strip_model()->ActivateTabAt(1, kIsUserGesture);
+  test_clock.Advance(base::TimeDelta::FromMinutes(1));
+  {
+    SCOPED_TRACE("");
+    ukm_entry_checker_->ExpectNewEntry(
+        kFOCEntryName, test_urls_[1],
+        {{TabManager_Background_ForegroundedOrClosed::kIsForegroundedName, 1},
+         {TabManager_Background_ForegroundedOrClosed::kLabelIdName, label_id_1},
+         {TabManager_Background_ForegroundedOrClosed::kIsDiscardedName, 0}});
+  }
+
+  // tab@2 is scored successfully.
+  base::Optional<float> tab_2 =
+      TabActivityWatcher::GetInstance()->CalculateReactivationScore(
+          browser()->tab_strip_model()->GetWebContentsAt(2), true);
+  EXPECT_TRUE(tab_2.has_value());
+  int64_t label_id_2 = 0;
+  {
+    SCOPED_TRACE("");
+    // tab feature of tab@2 should be logged correctly.
+    UkmMetricMap expected_tab_feature_values = kBasicMetricValues;
+    expected_tab_feature_values[TabManager_TabMetrics::kMRUIndexName] = 2;
+    expected_tab_feature_values
+        [TabManager_TabMetrics::kNumReactivationBeforeName] = 0;
+    expected_tab_feature_values[TabManager_TabMetrics::kTotalTabCountName] = 3;
+    expected_tab_feature_values[TabManager_TabMetrics::kWindowTabCountName] = 3;
+    expected_tab_feature_values[TabManager_TabMetrics::kQueryIdName] = query_id;
+
+    ukm_entry_checker_->ExpectNewEntry(kEntryName, test_urls_[2],
+                                       expected_tab_feature_values);
+    // Get label_id_2 for later verification.
+    label_id_2 =
+        GetLatestMetricValue(kEntryName, TabManager_TabMetrics::kLabelIdName);
+    EXPECT_NE(label_id_2, 0);
+  }
+
+  // No ForegroundedOrClosed event is logged for tab@1 because it's foregrounded
+  CloseBrowserSynchronously(browser());
+  {
+    SCOPED_TRACE("");
+    // Close Browser should log a ForegroundedOrClosed event for tab@2 with
+    // correct label_id.
+    ukm_entry_checker_->ExpectNewEntry(
+        kFOCEntryName, test_urls_[2],
+        {{TabManager_Background_ForegroundedOrClosed::kIsForegroundedName, 0},
+         {TabManager_Background_ForegroundedOrClosed::kLabelIdName, label_id_2},
+         {TabManager_Background_ForegroundedOrClosed::kIsDiscardedName, 0}});
+
+    // Close Browser should log a ForegroundedOrClosed event for tab@0 with
+    // label_id == 0 because  when we score it, it was not in the oldest N list.
+    ukm_entry_checker_->ExpectNewEntry(
+        kFOCEntryName, test_urls_[0],
+        {{TabManager_Background_ForegroundedOrClosed::kIsForegroundedName, 0},
+         {TabManager_Background_ForegroundedOrClosed::kLabelIdName, 0},
+         {TabManager_Background_ForegroundedOrClosed::kIsDiscardedName, 0}});
+  }
+}
+
+// Tests label id is recorded correctly for discarded tabs.
+IN_PROC_BROWSER_TEST_F(TabActivityWatcherUkmTest,
+                       DiscardedTabGetsCorrectLabelId) {
+  base::test::ScopedFeatureList scoped_feature_list;
+  scoped_feature_list.InitAndEnableFeatureWithParameters(
+      features::kTabRanker,
+      {{"number_of_oldest_tabs_to_log_with_TabRanker", "2"},
+       {"disable_background_log_with_TabRanker", "true"}});
+
+  ui_test_utils::NavigateToURL(browser(), test_urls_[0]);
+  AddTabAtIndex(1, test_urls_[1], ui::PAGE_TRANSITION_LINK);
+  // No TabMetrics events are logged till now.
+  EXPECT_EQ(0u, ukm_entry_checker_->NumEntries(kEntryName));
+
+  // Score tab_0 will succeed.
+  base::Optional<float> tab_0 =
+      TabActivityWatcher::GetInstance()->CalculateReactivationScore(
+          browser()->tab_strip_model()->GetWebContentsAt(0), true);
+  EXPECT_TRUE(tab_0.has_value());
+  int64_t label_id_0 = 0;
+  {
+    SCOPED_TRACE("");
+    UkmMetricMap expected_metrics = kBasicMetricValues;
+    expected_metrics[TabManager_TabMetrics::kNavigationEntryCountName] = 2;
+    // tab feature of tab@0 should be logged correctly.
+    ukm_entry_checker_->ExpectNewEntry(kEntryName, test_urls_[0],
+                                       expected_metrics);
+    // Get label_id_0 for later verification.
+    label_id_0 =
+        GetLatestMetricValue(kEntryName, TabManager_TabMetrics::kLabelIdName);
+    EXPECT_NE(label_id_0, 0);
+  }
+
+  // Discard the first tab.
+  content::WebContents* first_contents =
+      browser()->tab_strip_model()->GetWebContentsAt(0);
+  resource_coordinator::GetTabLifecycleUnitSource()
+      ->GetTabLifecycleUnitExternal(first_contents)
+      ->DiscardTab();
+
+  // Switching to first tab logs a forgrounded event for test_urls_[0].
+  browser()->tab_strip_model()->ActivateTabAt(0, kIsUserGesture);
+  {
+    SCOPED_TRACE("");
+    UkmMetricMap expected_metrics = {
+        {TabManager_Background_ForegroundedOrClosed::kIsForegroundedName, 1},
+        {TabManager_Background_ForegroundedOrClosed::kIsDiscardedName, 1},
+        {TabManager_Background_ForegroundedOrClosed::kLabelIdName, label_id_0}};
+
+    ukm_entry_checker_->ExpectNewEntry(kFOCEntryName, test_urls_[0],
+                                       expected_metrics);
+  }
+
+  // Score tab_1 will succeed.
+  base::Optional<float> tab_1 =
+      TabActivityWatcher::GetInstance()->CalculateReactivationScore(
+          browser()->tab_strip_model()->GetWebContentsAt(1), true);
+  EXPECT_TRUE(tab_1.has_value());
+  int64_t label_id_1 = 0;
+  {
+    SCOPED_TRACE("");
+    // tab feature of tab@1 should be logged correctly.
+    ukm_entry_checker_->ExpectNewEntry(kEntryName, test_urls_[1],
+                                       kBasicMetricValues);
+    // Get label_id_1 for later verification.
+    label_id_1 =
+        GetLatestMetricValue(kEntryName, TabManager_TabMetrics::kLabelIdName);
+    EXPECT_NE(label_id_0, 0);
+  }
+
+  // Discard the second tab.
+  content::WebContents* second_content =
+      browser()->tab_strip_model()->GetWebContentsAt(1);
+  resource_coordinator::GetTabLifecycleUnitSource()
+      ->GetTabLifecycleUnitExternal(second_content)
+      ->DiscardTab();
+
+  CloseBrowserSynchronously(browser());
+  {
+    SCOPED_TRACE("");
+    UkmMetricMap expected_metrics = {
+        {TabManager_Background_ForegroundedOrClosed::kIsForegroundedName, 0},
+        {TabManager_Background_ForegroundedOrClosed::kIsDiscardedName, 1},
+        {TabManager_Background_ForegroundedOrClosed::kLabelIdName, label_id_1}};
+
+    ukm_entry_checker_->ExpectNewEntry(kFOCEntryName, test_urls_[1],
+                                       expected_metrics);
+  }
+}
 }  // namespace resource_coordinator

chrome/browser/resource_coordinator/tab_lifecycle_unit.cc

@@ -446,15 +446,6 @@ base::ProcessHandle TabLifecycleUnitSource::TabLifecycleUnit::GetProcessHandle()
 
 LifecycleUnit::SortKey TabLifecycleUnitSource::TabLifecycleUnit::GetSortKey()
     const {
-  if (base::FeatureList::IsEnabled(features::kTabRanker)) {
-    base::Optional<float> reactivation_score =
-        resource_coordinator::TabActivityWatcher::GetInstance()
-            ->CalculateReactivationScore(web_contents());
-    if (reactivation_score.has_value())
-      return SortKey(reactivation_score.value(), last_focused_time_);
-    return SortKey(SortKey::kMaxScore, last_focused_time_);
-  }
-
   return SortKey(last_focused_time_);
 }
 

chrome/browser/resource_coordinator/tab_manager.cc

@@ -31,6 +31,7 @@
 #include "chrome/browser/memory/oom_memory_details.h"
 #include "chrome/browser/profiles/profile.h"
 #include "chrome/browser/resource_coordinator/background_tab_navigation_throttle.h"
+#include "chrome/browser/resource_coordinator/tab_activity_watcher.h"
 #include "chrome/browser/resource_coordinator/tab_lifecycle_unit_external.h"
 #include "chrome/browser/resource_coordinator/tab_manager.h"
 #include "chrome/browser/resource_coordinator/tab_manager_features.h"
@@ -97,6 +98,10 @@ struct LifecycleUnitAndSortKey {
       : lifecycle_unit(lifecycle_unit),
         sort_key(lifecycle_unit->GetSortKey()) {}
 
+  LifecycleUnitAndSortKey(LifecycleUnit* lifecycle_unit,
+                          const LifecycleUnit::SortKey& key)
+      : lifecycle_unit(lifecycle_unit), sort_key(key) {}
+
   bool operator<(const LifecycleUnitAndSortKey& other) const {
     return sort_key < other.sort_key;
   }
@@ -108,6 +113,23 @@ struct LifecycleUnitAndSortKey {
   LifecycleUnit::SortKey sort_key;
 };
 
+// Helper function that sorts |lifecycle_units_and_sort_keys|, and returns
+// a LifecycleUnitVector.
+LifecycleUnitVector SortedLifecycleUnitsFrom(
+    std::vector<LifecycleUnitAndSortKey>* lifecycle_units_and_sort_keys) {
+  std::sort(lifecycle_units_and_sort_keys->begin(),
+            lifecycle_units_and_sort_keys->end());
+
+  LifecycleUnitVector sorted_lifecycle_units;
+  sorted_lifecycle_units.reserve(lifecycle_units_and_sort_keys->size());
+  for (auto& lifecycle_unit_and_sort_key : *lifecycle_units_and_sort_keys) {
+    sorted_lifecycle_units.push_back(
+        lifecycle_unit_and_sort_key.lifecycle_unit);
+  }
+
+  return sorted_lifecycle_units;
+}
+
 std::unique_ptr<base::trace_event::ConvertableToTraceFormat> DataAsTraceValue(
     TabManager::BackgroundTabLoadingMode mode,
     size_t num_of_pending_navigations,
@@ -279,17 +301,8 @@ LifecycleUnitVector TabManager::GetSortedLifecycleUnits() {
   lifecycle_units_and_sort_keys.reserve(lifecycle_units_.size());
   for (auto* lifecycle_unit : lifecycle_units_)
     lifecycle_units_and_sort_keys.emplace_back(lifecycle_unit);
-  std::sort(lifecycle_units_and_sort_keys.begin(),
-            lifecycle_units_and_sort_keys.end());
-
-  LifecycleUnitVector sorted_lifecycle_units;
-  sorted_lifecycle_units.reserve(lifecycle_units_.size());
-  for (auto& lifecycle_unit_and_sort_key : lifecycle_units_and_sort_keys) {
-    sorted_lifecycle_units.push_back(
-        lifecycle_unit_and_sort_key.lifecycle_unit);
-  }
 
-  return sorted_lifecycle_units;
+  return SortedLifecycleUnitsFrom(&lifecycle_units_and_sort_keys);
 }
 
 void TabManager::DiscardTab(LifecycleUnitDiscardReason reason) {
@@ -657,7 +670,12 @@ content::WebContents* TabManager::DiscardTabImpl(
     LifecycleUnitDiscardReason reason) {
   DCHECK_CURRENTLY_ON(BrowserThread::UI);
 
-  for (LifecycleUnit* lifecycle_unit : GetSortedLifecycleUnits()) {
+  const LifecycleUnitVector sorted_lifecycle_units =
+      base::FeatureList::IsEnabled(features::kTabRanker)
+          ? GetSortedLifecycleUnitsFromTabRanker()
+          : GetSortedLifecycleUnits();
+
+  for (LifecycleUnit* lifecycle_unit : sorted_lifecycle_units) {
     DecisionDetails decision_details;
     if (lifecycle_unit->CanDiscard(reason, &decision_details) &&
         lifecycle_unit->Discard(reason)) {
@@ -1160,4 +1178,34 @@ bool TabManager::ShouldProactivelyDiscardTabs() {
   return true;
 }
 
+LifecycleUnitVector TabManager::GetSortedLifecycleUnitsFromTabRanker() {
+  // Set query_id if TabRanker is enabled.
+  const int64_t query_id = static_cast<int64_t>(base::RandUint64());
+  resource_coordinator::TabActivityWatcher::GetInstance()
+      ->SetQueryIdForTabMetricsLogger(query_id);
+
+  std::vector<LifecycleUnitAndSortKey> lifecycle_units_and_sort_keys;
+  lifecycle_units_and_sort_keys.reserve(lifecycle_units_.size());
+  for (auto* lifecycle_unit : lifecycle_units_) {
+    TabLifecycleUnitExternal* tab_lifecycle_unit_external =
+        lifecycle_unit->AsTabLifecycleUnitExternal();
+    // For now, all LifecycleUnits are TabLifecycleUnitExternals.
+    DCHECK(tab_lifecycle_unit_external);
+
+    base::Optional<float> reactivation_score =
+        resource_coordinator::TabActivityWatcher::GetInstance()
+            ->CalculateReactivationScore(
+                tab_lifecycle_unit_external->GetWebContents(), true);
+
+    float score = reactivation_score.has_value()
+                      ? reactivation_score.value()
+                      : LifecycleUnit::SortKey::kMaxScore;
+    lifecycle_units_and_sort_keys.emplace_back(
+        lifecycle_unit,
+        LifecycleUnit::SortKey(score, lifecycle_unit->GetLastFocusedTime()));
+  }
+
+  return SortedLifecycleUnitsFrom(&lifecycle_units_and_sort_keys);
+}
+
 }  // namespace resource_coordinator

chrome/browser/resource_coordinator/tab_manager.h

@@ -471,6 +471,13 @@ class TabManager : public LifecycleUnitObserver,
   // Indicates if TabManager should proactively discard tabs.
   bool ShouldProactivelyDiscardTabs();
 
+  // Returns the LifecycleUnits managed by this, similar to
+  // GetSortedLifecycleUnits. The result is sorted based on
+  // (reactivation_score_from_tab_ranker, last_activity_time).
+  // TODO(charleszhao): This is only used for TabRanker experiment and should be
+  // cleaned up once the experiment is done. https://crbug.com/908959.
+  LifecycleUnitVector GetSortedLifecycleUnitsFromTabRanker();
+
   // LifecycleUnits managed by this.
   LifecycleUnitSet lifecycle_units_;
 

chrome/browser/resource_coordinator/tab_manager_features.cc

@@ -306,4 +306,14 @@ int GetNumOldestTabsToScoreWithTabRanker() {
       std::numeric_limits<int>::max());
 }
 
+int GetNumOldestTabsToLogWithTabRanker() {
+  return base::GetFieldTrialParamByFeatureAsInt(
+      features::kTabRanker, "number_of_oldest_tabs_to_log_with_TabRanker", 0);
+}
+
+bool DisableBackgroundLogWithTabRanker() {
+  return base::GetFieldTrialParamByFeatureAsBool(
+      features::kTabRanker, "disable_background_log_with_TabRanker", false);
+}
+
 }  // namespace resource_coordinator

chrome/browser/resource_coordinator/tab_manager_features.h

@@ -329,6 +329,12 @@ InfiniteSessionRestoreParams GetInfiniteSessionRestoreParams();
 // Gets number of oldest tab that should be scored by TabRanker.
 int GetNumOldestTabsToScoreWithTabRanker();
 
+// Gets number of oldest tabs that should be logged by TabRanker.
+int GetNumOldestTabsToLogWithTabRanker();
+
+// Whether to disable background time TabMetrics log.
+bool DisableBackgroundLogWithTabRanker();
+
 }  // namespace resource_coordinator
 
 #endif  // CHROME_BROWSER_RESOURCE_COORDINATOR_TAB_MANAGER_FEATURES_H_

chrome/browser/resource_coordinator/tab_metrics_logger.cc

@@ -143,7 +143,8 @@ tab_ranker::TabFeatures TabMetricsLogger::GetTabFeatures(
 void TabMetricsLogger::LogTabMetrics(
     ukm::SourceId ukm_source_id,
     const tab_ranker::TabFeatures& tab_features,
-    content::WebContents* web_contents) {
+    content::WebContents* web_contents,
+    int64_t label_id) {
   if (!ukm_source_id)
     return;
 
@@ -167,7 +168,8 @@ void TabMetricsLogger::LogTabMetrics(
 
   ukm::builders::TabManager_TabMetrics entry(ukm_source_id);
   PopulateTabFeaturesToUkmEntry(tab_features, &entry);
-  entry.SetSequenceId(++sequence_id_);
+  entry.SetLabelId(label_id);
+  entry.SetQueryId(query_id_);
   entry.Record(ukm::UkmRecorder::Get());
 }
 
@@ -178,7 +180,7 @@ void TabMetricsLogger::LogForegroundedOrClosedMetrics(
     return;
 
   ukm::builders::TabManager_Background_ForegroundedOrClosed(ukm_source_id)
-      .SetSequenceId(++sequence_id_)
+      .SetLabelId(metrics.label_id)
       .SetIsForegrounded(metrics.is_foregrounded)
       .SetMRUIndex(metrics.mru_index)
       .SetTimeFromBackgrounded(metrics.time_from_backgrounded)

chrome/browser/resource_coordinator/tab_metrics_logger.h

@@ -60,6 +60,7 @@ class TabMetricsLogger {
     int64_t time_from_backgrounded = 0;
     int mru_index = 0;
     int total_tab_count = 0;
+    int64_t label_id = 0;
   };
 
   TabMetricsLogger();
@@ -69,7 +70,8 @@ class TabMetricsLogger {
   // |ukm_source_id| is zero.
   void LogTabMetrics(ukm::SourceId ukm_source_id,
                      const tab_ranker::TabFeatures& tab_features,
-                     content::WebContents* web_contents);
+                     content::WebContents* web_contents,
+                     int64_t label_id);
 
   // Logs TabManager.Background.ForegroundedOrClosed UKM for a tab that was
   // shown or closed after being inactive.
@@ -98,10 +100,12 @@ class TabMetricsLogger {
       const TabMetrics& tab_metrics,
       base::TimeDelta inactive_duration);
 
+  void set_query_id(int64_t query_id) { query_id_ = query_id; }
+
  private:
-  // A counter to be incremented and logged with each UKM entry, used to
-  // indicate the order that events within the same report were logged.
-  int sequence_id_ = 0;
+  // query_id should be set whenever a new tabRanker query happens, so all logs
+  // that happened within the same query will have same query_id_.
+  int64_t query_id_ = 0;
 
   DISALLOW_COPY_AND_ASSIGN(TabMetricsLogger);
 };

chrome/browser/resource_coordinator/tab_metrics_logger_unittest.cc

@@ -168,8 +168,11 @@ class TabMetricsLoggerUKMTest : public ::testing::Test {
 TEST_F(TabMetricsLoggerUKMTest, LogTabMetrics) {
   const tab_ranker::TabFeatures tab =
       tab_ranker::GetFullTabFeaturesForTesting();
+  const int64_t query_id = 1234;
+  const int64_t label_id = 5678;
+  GetLogger()->set_query_id(query_id);
 
-  GetLogger()->LogTabMetrics(GetSourceId(), tab, nullptr);
+  GetLogger()->LogTabMetrics(GetSourceId(), tab, nullptr, label_id);
 
   // Checks that the size is logged correctly.
   EXPECT_EQ(1U, GetTestUkmRecorder()->sources_count());
@@ -184,6 +187,7 @@ TEST_F(TabMetricsLoggerUKMTest, LogTabMetrics) {
                                 {"HasFormEntry", 1},
                                 {"IsPinned", 1},
                                 {"KeyEventCount", 21},
+                                {"LabelId", label_id},
                                 {"MouseEventCount", 22},
                                 {"MRUIndex", 27},
                                 {"NavigationEntryCount", 24},
@@ -191,7 +195,7 @@ TEST_F(TabMetricsLoggerUKMTest, LogTabMetrics) {
                                 {"PageTransitionCoreType", 2},
                                 {"PageTransitionFromAddressBar", 1},
                                 {"PageTransitionIsRedirect", 1},
-                                {"SequenceId", 1},
+                                {"QueryId", query_id},
                                 {"SiteEngagementScore", 26},
                                 {"TimeFromBackgrounded", 10000},
                                 {"TotalTabCount", 30},
@@ -212,6 +216,7 @@ TEST_F(TabMetricsLoggerUKMTest, LogForegroundedOrClosedMetrics) {
   foc_metrics.time_from_backgrounded = 1234;
   foc_metrics.mru_index = 4;
   foc_metrics.total_tab_count = 7;
+  foc_metrics.label_id = 5678;
 
   GetLogger()->LogForegroundedOrClosedMetrics(GetSourceId(), foc_metrics);
 
@@ -225,30 +230,12 @@ TEST_F(TabMetricsLoggerUKMTest, LogForegroundedOrClosedMetrics) {
 
   // Checks that all the fields are logged correctly.
   ExpectEntries(entries[0], {
-                                {"SequenceId", 1},
-                                {"IsForegrounded", 0},
+                                {"IsDiscarded", foc_metrics.is_discarded},
+                                {"IsForegrounded", foc_metrics.is_foregrounded},
+                                {"LabelId", foc_metrics.label_id},
                                 {"MRUIndex", foc_metrics.mru_index},
                                 {"TimeFromBackgrounded",
                                  foc_metrics.time_from_backgrounded},
                                 {"TotalTabCount", foc_metrics.total_tab_count},
-                                {"IsDiscarded", foc_metrics.is_discarded},
                             });
 }
-
-// Checks the sequence id is logged as sequentially incremental sequence across
-// different events.
-TEST_F(TabMetricsLoggerUKMTest, SequenceIdShouldBeLoggedSequentially) {
-  const TabMetricsLogger::ForegroundedOrClosedMetrics foc_metrics;
-
-  GetLogger()->LogForegroundedOrClosedMetrics(GetSourceId(), foc_metrics);
-  GetLogger()->LogForegroundedOrClosedMetrics(GetSourceId(), foc_metrics);
-
-  EXPECT_EQ(2U, GetTestUkmRecorder()->sources_count());
-  EXPECT_EQ(2U, GetTestUkmRecorder()->entries_count());
-  const std::vector<const ukm::mojom::UkmEntry*> entries =
-      GetTestUkmRecorder()->GetEntriesByName(
-          "TabManager.Background.ForegroundedOrClosed");
-  EXPECT_EQ(2U, entries.size());
-  GetTestUkmRecorder()->ExpectEntryMetric(entries[0], "SequenceId", 1);
-  GetTestUkmRecorder()->ExpectEntryMetric(entries[1], "SequenceId", 2);
-}

tools/metrics/ukm/ukm.xml

@@ -4531,6 +4531,12 @@ be describing additional metrics about the same event.
       brought to foreground again.
     </summary>
   </metric>
+  <metric name="LabelId">
+    <summary>
+      An int64 random number generated when TabManager.TabMetric is logged. The
+      same number is also logged here, so that feature label can be paired.
+    </summary>
+  </metric>
   <metric name="MRUIndex">
     <summary>
       Index of the tab in most-recently-used order. A value of N means there are
@@ -4539,10 +4545,14 @@ be describing additional metrics about the same event.
     </summary>
   </metric>
   <metric name="SequenceId">
+    <obsolete>
+      Deprecated 11/2018 in favor LabelId.
+    </obsolete>
     <summary>
-      The sequence of this event and TabManager.TabMetrics event in the current
-      session. Incremented by 1 each time one of the two events is logged to
-      provide an ordering of events.
+      This metric is obsolete in 11/2018 in favor of LabelId for pairing with
+      Tabmanager.TabMetrics event. The sequence of this event and
+      TabManager.TabMetrics event in the current session. Incremented by 1 each
+      time one of the two events is logged to provide an ordering of events.
     </summary>
   </metric>
   <metric name="TimeFromBackgrounded">
@@ -5338,6 +5348,13 @@ be describing additional metrics about the same event.
       Number of key events that were sent to the page.
     </summary>
   </metric>
+  <metric name="LabelId">
+    <summary>
+      An int64 random number generated at logging time. The same number will
+      also be logged to the TabManager.Background.ForegroundedOrClosed event, so
+      that feature label can be paired.
+    </summary>
+  </metric>
   <metric name="MouseEventCount">
     <summary>
       Number of mouse events that were sent to the page.
@@ -5379,11 +5396,23 @@ be describing additional metrics about the same event.
       PageTransition type may not be accurate.
     </summary>
   </metric>
+  <metric name="QueryId">
+    <summary>
+      An int64 random number generated at query time of
+      TabManager::GetSortedLifecycleUnitsFromTabRanker. Tabs with the same
+      QueryId will be later on combined in one full list for further analysis.
+    </summary>
+  </metric>
   <metric name="SequenceId">
+    <obsolete>
+      Deprecated 11/2018 in favor LabelId.
+    </obsolete>
     <summary>
-      The sequence of this event and TabManager.ForegroundedOrClosed event in
-      the current session. Incremented by 1 each time one of the two events is
-      logged to provide an ordering of events.
+      This metric is obsolete in 11/2018 in favor of LabelId for pairing with
+      Tabmanager.ForegroundedOrClosed event. The sequence of this event and
+      TabManager.ForegroundedOrClosed event in the current session. Incremented
+      by 1 each time one of the two events is logged to provide an ordering of
+      events.
     </summary>
   </metric>
   <metric name="SiteEngagementScore">