Media classes for local learning.

This CL contains some (unused) classes that form the basis for the
media local learning experiment.

These are in the media::learning namespace, rather than just media,
for two reasons:
  - Some of the names ("Value") are pretty generic.
  - I'd like to move it to components/ if it's generally useful.

Bug: 895929
Change-Id: I92f2f23672d0256449b39da4d32c6b4e1595c5c8
Reviewed-on: https://chromium-review.googlesource.com/c/1283992
Commit-Queue: Frank Liberato <liberato@chromium.org>
Reviewed-by: Xiaohan Wang <xhwang@chromium.org>
Cr-Commit-Position: refs/heads/master@{#600847}

media/BUILD.gn

@@ -144,6 +144,7 @@ test("media_unittests") {
     "//media/filters:unit_tests",
     "//media/formats:unit_tests",
     "//media/gpu:unit_tests",
+    "//media/learning:unit_tests",
     "//media/mojo:unit_tests",
     "//media/muxers:unit_tests",
     "//media/renderers:unit_tests",

media/learning/BUILD.gn

+# Copyright 2018 The Chromium Authors. All rights reserved.
+# Use of this source code is governed by a BSD-style license that can be
+# found in the LICENSE file.
+
+source_set("learning") {
+  # Do not expand the visibility here without double-checking with OWNERS, this
+  # is a roll-up target which is part of the //media component. Most other DEPs
+  # should be using //media and not directly DEP this roll-up target.
+  visibility = [ "//media" ]
+
+  sources = [
+    "instance.cc",
+    "instance.h",
+    "learner.h",
+    "learner_factory.h",
+    "learning_task.cc",
+    "learning_task.h",
+    "value.cc",
+    "value.h",
+  ]
+
+  public_deps = [
+    "//base",
+    "//media/base",
+  ]
+
+  configs += [ "//media:subcomponent_config" ]
+}
+
+source_set("unit_tests") {
+  testonly = true
+  sources = [
+    "value_unittest.cc",
+  ]
+
+  deps = [
+    "//base/test:test_support",
+    "//media:test_support",
+    "//testing/gtest",
+  ]
+}

media/learning/README.md

+# Media Local Learning Framework
+
+This directory contains code to support media's local learning experiment.
+It provides lightweight learning algorithms that can be trained on the user's
+local device to tailor media performance / behavior to match the user's usage.
+
+## Terms
+
+**Feature vector** - How we describe the "state of the world" to the learner.
+  For example, we might describe a video using the features {width, height,
+  format, frame rate}.
+
+**Target** - The output value we'd like to predict, given some features.  We
+  might want to know a boolean representing "will playback be smooth?".
+
+**Training example** - A tuple {Feature vector, Target value} that demonstrates
+  the desired target for some feature vector. The learning algorithm collects
+  examples, and tries to generalize them to unseen features.
+
+**Classification** - Class of problems for which the target value is nominal.
+  For example, predicting the expected color from a set of five colors is
+  a classification task.  The key idea is that the target values are unordered.
+
+**Regression** - Class of problems for which the target value is numeric.  For
+  example, predicting how tall a plant will grow is regression.
+
+**Model** - A class of functions that relates features (inputs) to target values
+  (outputs).  For example, a linear model relates them as:
+  ```
+  target = weight1 * feature1 + weight2 * feature2 + ...
+  ```
+  Note that the weights aren't known in advance; we'll choose them as part of
+  the training process based on the training examples.
+
+**Model parameters** - The missing values in our model that the learning
+  learning algorithm tries to figure out based on the training data.  In our
+  linear model, we'd need to know `weight1` and `weight2`.
+
+**Learning task** - A problem we're trying to solve.  For example, "Will this
+  video element be played before it's destroyed?"
+
+## Classes
+
+There are several classes that we define here.  While more detail can generally
+be found in the header for the class, an overview of the main ones is:
+
+**Learner** - Base class for a thing that knows how to convert training data
+  into a fully trained model (model + parameters).  For example, we might have
+  a Learner subclass that chooses the parameters for a Naive Bayes model.
+  Similarly, we might have a Learner that trains a linear regression model.
+
+**LearningTask** - Description of a task, and also, because it's convenient,
+  a choice of model that will be used to learn it.  It contains:
+
+    * name
+    * description of features (name, nominal vs numeric, etc.)
+    * description of the target value
+    * description / parameters of the learning model to be used
+
+**Instance** - Set of feature values.
+
+**Value** - Representation of a number or (hashed) string.
+
+## Models
+
+All of our models are supervised.

media/learning/instance.cc

+// Copyright 2018 The Chromium Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file.
+
+#include "media/learning/instance.h"
+
+namespace media {
+namespace learning {
+
+Instance::Instance() = default;
+Instance::~Instance() = default;
+Instance::Instance(Instance&& rhs) {
+  features = std::move(rhs.features);
+}
+
+std::ostream& operator<<(std::ostream& out, const Instance& instance) {
+  for (const auto& feature : instance.features)
+    out << " " << feature;
+
+  return out;
+}
+
+}  // namespace learning
+}  // namespace media

media/learning/instance.h

+// Copyright 2018 The Chromium Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file.
+
+#ifndef MEDIA_LEARNING_INSTANCE_H_
+#define MEDIA_LEARNING_INSTANCE_H_
+
+#include <ostream>
+#include <vector>
+
+#include "base/macros.h"
+#include "media/base/media_export.h"
+#include "media/learning/value.h"
+
+namespace media {
+namespace learning {
+
+// One instance == group of feature values.
+struct MEDIA_EXPORT Instance {
+  Instance();
+  ~Instance();
+  // Declare a no-exception move constructor so that std::vector will use it.
+  Instance(Instance&& rhs) noexcept;
+
+  // It's up to you to add the right number of features to match the learner
+  // description.  Otherwise, the learner will ignore (training) or lie to you
+  // (inference), silently.
+  std::vector<FeatureValue> features;
+
+  DISALLOW_COPY_AND_ASSIGN(Instance);
+};
+
+MEDIA_EXPORT std::ostream& operator<<(std::ostream& out,
+                                      const Instance& instance);
+
+}  // namespace learning
+}  // namespace media
+
+#endif  // MEDIA_LEARNING_INSTANCE_H_

media/learning/learner.h

+// Copyright 2018 The Chromium Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file.
+
+#ifndef MEDIA_LEARNING_LEARNER_H_
+#define MEDIA_LEARNING_LEARNER_H_
+
+#include "base/values.h"
+#include "media/base/media_export.h"
+#include "media/learning/instance.h"
+
+namespace media {
+namespace learning {
+
+// Base class for a thing that takes examples of the form {features, target},
+// and trains a model to predict the target given the features.  The target may
+// be either nominal (classification) or numeric (regression), though this must
+// be chosen in advance when creating the learner via LearnerFactory.
+class MEDIA_EXPORT Learner {
+ public:
+  virtual ~Learner() = default;
+
+  // Tell the learner that |instance| has been observed with the target value
+  // |target| during training.
+  virtual void AddExample(const Instance& instance,
+                          const TargetValue& target) = 0;
+};
+
+}  // namespace learning
+}  // namespace media
+
+#endif  // MEDIA_LEARNING_LEARNER_H_

media/learning/learner_factory.h

+// Copyright 2018 The Chromium Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file.
+
+#ifndef MEDIA_LEARNING_LEARNER_FACTORY_H_
+#define MEDIA_LEARNING_LEARNER_FACTORY_H_
+
+#include <string>
+
+#include "media/base/media_export.h"
+#include "media/learning/learing_task.h"
+#include "media/learning/learner.h"
+
+namespace media {
+namespace learning {
+
+// Factory class for learner instances.
+class MEDIA_EXPORT LearnerFactory {
+ public:
+  virtual ~LearnerFactory() = default;
+
+  // Provide a learner that matches |task|.
+  virtual std::unique_ptr<Learner> CreateLearner(const LearningTask& task) = 0;
+};
+
+}  // namespace learning
+}  // namespace media
+
+#endif  // MEDIA_LEARNING_LEARNER_FACTORY_H_

media/learning/learning_task.cc

+// Copyright 2018 The Chromium Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file.
+
+#include "media/learning/learning_task.h"
+
+namespace media {
+namespace learning {
+
+LearningTask::LearningTask() = default;
+LearningTask::LearningTask(const LearningTask&) = default;
+LearningTask::~LearningTask() = default;
+
+}  // namespace learning
+}  // namespace media

media/learning/learning_task.h

+// Copyright 2018 The Chromium Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file.
+
+#ifndef MEDIA_LEARNING_LEARNING_TASK_H_
+#define MEDIA_LEARNING_LEARNING_TASK_H_
+
+#include <initializer_list>
+#include <string>
+
+#include "media/base/media_export.h"
+#include "media/learning/instance.h"
+
+namespace media {
+namespace learning {
+
+// Description of a learning task.  This includes both the description of the
+// inputs (features) and output (target value), plus a choice of the model and
+// parameters for learning.
+// TODO(liberato): Consider separating the task from the choice of model.
+struct MEDIA_EXPORT LearningTask {
+  // Not all models support all feature / target descriptions.  For example,
+  // NaiveBayes requires kUnordered features.  Similarly, kLogLinear doesn't
+  // support kUnordered features or targets.  kRandomForest might support more
+  // combination of orderings and types.
+  //
+  // Also note that not all of these are implemented yet.
+  enum class Model {
+    kMostCommonTarget,
+    kNaiveBayes,
+    kRandomForest,
+    kLogLinear,
+  };
+
+  enum class Ordering {
+    // Values are not ordered; nearby values might have wildly different
+    // meanings.  For example, two ints that are computed by taking the hash
+    // of a string are unordered; it's categorical data.  Values of type DOUBLE
+    // should almost certainly not be kUnordered; discretize them in some way
+    // if you really want to make discrete, unordered buckets out of them.
+    kUnordered,
+
+    // Values may be interpreted as being in numeric order.  For example, two
+    // ints that represent the number of elapsed milliseconds are numerically
+    // ordered in a meaningful way.
+    kNumeric,
+  };
+
+  enum class PrivacyMode {
+    // Value represents private information, such as a URL that was visited by
+    // the user.
+    kPrivate,
+
+    // Value does not represent private information, such as video width.
+    kPublic,
+  };
+
+  // Description of how a Value should be interpreted.
+  struct ValueDescription {
+    // Name of this value, such as "source_url" or "width".
+    std::string name;
+
+    // Is this value nominal or not?
+    Ordering ordering = Ordering::kUnordered;
+
+    // Should this value be treated as being private?
+    PrivacyMode privacy_mode = PrivacyMode::kPublic;
+  };
+
+  LearningTask();
+  LearningTask(const std::string& name,
+               Model model,
+               std::initializer_list<ValueDescription> feature_init_list,
+               ValueDescription target_description);
+  LearningTask(const LearningTask&);
+  ~LearningTask();
+
+  // Unique name for this learner.
+  std::string name;
+
+  Model model = Model::kMostCommonTarget;
+
+  std::vector<ValueDescription> feature_descriptions;
+
+  // Note that kUnordered targets indicate classification, while kOrdered
+  // targes indicate regression.
+  ValueDescription target_description;
+
+  // TODO(liberato): add training parameters, like smoothing constants.  It's
+  // okay if some of these are model-specific.
+};
+
+}  // namespace learning
+}  // namespace media
+
+#endif  // MEDIA_LEARNING_LEARNING_TASK_H_

media/learning/value.cc

+// Copyright 2018 The Chromium Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file.
+
+#include "media/learning/value.h"
+
+#include "base/hash.h"
+
+namespace media {
+namespace learning {
+
+Value::Value(int x) : value_(x) {}
+Value::Value(const char* x) {
+  // std::hash would be nice, but it can (and does) change values between
+  // different instances of the class.  In other words, Value("foo") !=
+  // Value("foo") necessarily.
+  if (x)
+    value_ = base::PersistentHash(x, strlen(x));
+}
+
+Value::Value(const std::string& x) : value_(base::PersistentHash(x)) {}
+
+Value::Value(const Value& other) : value_(other.value_) {}
+
+bool Value::operator==(const Value& rhs) const {
+  return value_ == rhs.value_;
+}
+
+bool Value::operator!=(const Value& rhs) const {
+  return value_ != rhs.value_;
+}
+
+bool Value::operator<(const Value& rhs) const {
+  return value_ < rhs.value_;
+}
+
+std::ostream& operator<<(std::ostream& out, const Value& value) {
+  return out << value.value_;
+}
+
+}  // namespace learning
+}  // namespace media

media/learning/value.h

+// Copyright 2018 The Chromium Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file.
+
+#ifndef MEDIA_LEARNING_VALUE_H_
+#define MEDIA_LEARNING_VALUE_H_
+
+#include <cstdint>
+#include <ostream>
+#include <string>
+
+#include "media/base/media_export.h"
+
+namespace media {
+namespace learning {
+
+// Lightweight, copyable value for features and labels.
+// Strings are hashed into ints in an implementation-specific way, so don't
+// count on ordering between them.  See LearningTask for more info about nominal
+// versus numerical values.
+//
+// For numeric values, ==, !=, > operators behave as one would expect.
+//
+// For strings, only == and != are guaranteed to be meaningful.
+class MEDIA_EXPORT Value {
+ public:
+  explicit Value(int x);
+  explicit Value(const char* x);
+  explicit Value(const std::string& x);
+
+  Value(const Value& other);
+
+  bool operator==(const Value& rhs) const;
+  bool operator!=(const Value& rhs) const;
+  bool operator<(const Value& rhs) const;
+
+ private:
+  int64_t value_ = 0;
+
+  friend MEDIA_EXPORT std::ostream& operator<<(std::ostream& out,
+                                               const Value& value);
+
+  // Copy and assign are fine.
+};
+
+// Just to make it clearer what type of value we mean in context.
+using FeatureValue = Value;
+using TargetValue = Value;
+
+MEDIA_EXPORT std::ostream& operator<<(std::ostream& out, const Value& value);
+
+}  // namespace learning
+}  // namespace media
+
+#endif  // MEDIA_LEARNING_VALUE_H_

media/learning/value_unittest.cc

+// Copyright 2018 The Chromium Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file.
+
+#include "media/learning/value.h"
+
+#include "testing/gtest/include/gtest/gtest.h"
+
+namespace media {
+namespace learning {
+
+class LearnerValueTest : public testing::Test {};
+
+TEST_F(LearnerValueTest, EqualStringsCompareAsEqual) {
+  const char* value = "value";
+  Value v1(value);
+  Value v2(value);
+  EXPECT_TRUE(v1 == v2);
+}
+
+TEST_F(LearnerValueTest, UnequalStringsDoNotCompareAsEqual) {
+  Value v1("value");
+  Value v2("nonvalue");
+  EXPECT_TRUE(v1 != v2);
+}
+
+TEST_F(LearnerValueTest, ConstCharAndStdStringsCompareAsEqual) {
+  const char* value_as_char = "value";
+  std::string value_as_str(value_as_char);
+  Value v1(value_as_char);
+  Value v2(value_as_str);
+  EXPECT_TRUE(v1 == v2);
+}
+
+TEST_F(LearnerValueTest, IntsCompareCorrectly) {
+  const int i1 = 12345;
+  const int i2 = 67890;
+  Value v1(i1);
+  Value v2(i1);
+  Value v3(i2);
+  EXPECT_TRUE(v1 == v2);
+  EXPECT_TRUE(v1 != v3);
+  EXPECT_TRUE(v1 < v3);
+  EXPECT_FALSE(v3 < v1);
+}
+
+}  // namespace learning
+}  // namespace media

media/media_options.gni

@@ -21,6 +21,7 @@ media_subcomponent_deps = [
   "//media/device_monitors",
   "//media/filters",
   "//media/formats",
+  "//media/learning",
   "//media/muxers",
   "//media/renderers",
   "//media/video",