[LevelDBScopes] Creating lock manager

This is the first component of LevelDB Scopes. The lock manager is a
basic leveled lock system with shared and exclusive locks. This change
contains an implementation which requires disjoint lock ranges, which
is what the IndexedDB system will be using.

This also includes a README with the implementation plan. This will be
updated as more code is submitted.

Bug: 862456
Change-Id: Ib4300ae8c59035c06772625323b3de0cc9ef3f6a
Reviewed-on: https://chromium-review.googlesource.com/1134407
Commit-Queue: Daniel Murphy <dmurph@chromium.org>
Reviewed-by: John Abd-El-Malek <jam@chromium.org>
Reviewed-by: Victor Costan <pwnall@chromium.org>
Cr-Commit-Position: refs/heads/master@{#581385}

content/browser/BUILD.gn

@@ -983,6 +983,10 @@ jumbo_source_set("browser") {
     "indexed_db/leveldb/leveldb_write_batch.cc",
     "indexed_db/leveldb/leveldb_write_batch.h",
     "indexed_db/list_set.h",
+    "indexed_db/scopes/disjoint_range_lock_manager.cc",
+    "indexed_db/scopes/disjoint_range_lock_manager.h",
+    "indexed_db/scopes/scopes_lock_manager.cc",
+    "indexed_db/scopes/scopes_lock_manager.h",
     "initiator_csp_context.cc",
     "initiator_csp_context.h",
     "installedapp/installed_app_provider_impl_default.cc",

content/browser/indexed_db/scopes/README.md

+# LevelDB Scopes Implementation Design
+
+This document is the implementation plan and design for LevelDB Scopes. It serves to both document the current state and the future plans for implementing LevelDB Scopes.
+
+See LevelDB Scopes general design doc [here](https://docs.google.com/document/d/16_igCI15Gfzb6UYqeuJTmJPrzEtawz6Y1tVOKNtYgiU/edit).
+
+# Status / Current State
+
+The only thing implemented so far is the lock manager interface & descrete implementation.
+
+# Vocabulary
+
+**Scope**
+
+A scope encompases a group of changes that can be reverted. It is basically synonymous with a transaction, and would be used for readwrite and versionchange transactions in LevelDB. The scope has a defined list of key ranges where the changes can occur. It operates by keeping an undo log, which is either discarded on commit, or replayed on revert.
+
+**Undo Log**
+
+Each scope saves an undo log, which has all the information needed to undo all changes performed in the scope.
+
+**Scope Number (scope#)**
+
+Each scope has an identifier unique to the backing store that is auto-incrementing. During recovery, it is set to the minimum unused scope (see more in the recovery section).
+
+**Sequence Number (seq#)**
+
+Every undo log entry has a unique sequence number within the scope. These should start at <max int> (using Fixed64) and decrement. The sequence number '0' is reserved for the commit point
+
+**Commit Point**
+
+This signifies that a scope has been committed. It also means that a specific sequence number was written for a scope.
+
+**Key Range**
+
+Represents a range of LevelDB keys. Every operation has a key or a key range associated with it. Sometimes key ranges are expected to be re-used or modified again by subsequent operations, and sometimes key ranges are known to be never used again.
+
+**Lock**
+
+To prevent reading uncommitted data, IndexedDB 'locks' objects stores when there are (readwrite) transactions operating on them.
+
+## New LevelDB Table Data
+
+|Purpose|Key |Final format|Value (protobufs)|
+|---|---|---|---|
+|Metadata|undo-metadata|`prefix-0`|`{version: 1}`|
+|Scope Metadata|undo-scopes-<scope#>|`prefix-1-<scope#>`|key ranges if scope is active, or \<empty\> if committed.|
+|Undo log operations|undo-log-<scope#>-<seq#>|`prefix-2-<scope#>-<seq#>`|undo operation|
+|Commit point|undo-log-<scope#>-0|`prefix-2-<scope#>-0`| \<empty\> OR \<ranges to delete\> |
+
+
+### Key Format
+
+* Allow the 'user' of the scopes system to choose the key prefix (`prefix`).
+* Scope # is a varint
+* Sequence # is a Fixed64
+
+### Value Format
+
+All values will be protobufs
+
+# Operation Flows
+
+## Creating & Using a Scope
+IndexedDB Sequence
+
+* Input - key ranges for the scope. Eg - the applicable object stores (and indexes) for a readwrite txn, or a whole database for a versionchange txn.
+* If any of the key ranges are currently locked, then wait until they are free. This would replace the IDB transaction sequencing logic.
+* Create the scope 
+    * Use the next available scope # (and increment the next scope #)
+    * Signal the locks service that the key ranges for this scope are now locked
+    * Write undo-scopes-scope# -> key_ranges to LevelDB
+* Enable operations on the scope (probably eventually by binding it using mojo)
+* For every operation, the scope must read the database to generate the undo log
+    * See the [Undo Operation Generation](#undo-operations) section below
+* Output - a Scope
+
+## Committing a Scope
+**IndexedDB Sequence**
+
+* Input - a Scope
+* The scope is marked as 'committed', the commit point is written to the undo log (undo-scope#-0), and the metadatais updated to remove the key ranges (undo-scopes-scope# -> <empty>). This change is flushed to disk.
+* The Cleanup & Revert Sequence is signalled for cleaning up the committed scope #.
+* Output - Scope is committed, and lock is released.
+
+## Reverting a Scope
+**Cleanup & Revert Sequence**
+
+* Input - revert a given scope number.
+* Opens a cursor to undo-<scope#>-0
+    * Cursor should be at the first undo entry
+    * If sequence #0 exists (this key exists), then error - reverting a committed scope is an invalid state in this design
+* Iterate through undo log, commiting operations and deleting each undo entry.
+* Delete the lock entry at undo-scopes-<scope#>, and flush this change to disk.
+* Output - Scope was successfully reverted, and lock released.
+
+## Startup & Recovery
+**IndexedDB Sequence**
+
+* Input - the database
+* Reads undo-metadata (fail for unknown version)
+* Opens a cursor to undo-scopes- & iterates to read in all scopes
+    * If the scope metadata has key ranges, then those are considered locked
+    * The maximum scope # is found and used to determine the next scope number (used in scope creation)
+* Signals the lock system which locks are held
+* Signals IndexedDB that it can start accepting operations (IndexedDB can now start running)
+* For every undo-scopes- metadata that is empty
+    * Kick off an [Undo Log Cleanup](#undo-log-cleanup) task to eventually clean this up.
+* For every undo-scopes- metadata with key ranges
+    * If there is a commit point for any of these scopes, then that is an invalid state / corruption (as the scopes value should have been emptied in the same writebatch that wrote the commit point).
+    * Kick off a [Reverting a Scope](#reverting-a-scope) task. This state indicates a shutdown while a revert was in progress.
+* Output - nothing, done
+
+## Undo Log Cleanup
+**Cleanup & Revert Sequence**
+
+* Input - nothing
+* Scan the undo- namespace for commit points. If a undo-scope#-0 is found, then start deleting that undo log
+    * This can happen in multiple write batches
+* If ranges are found in the undo-scope#-0 (commit point), then
+    * Those ranges are also deleted, in batches.
+    * Possibly compact these ranges at the end.
+* The undo-scope#-0 and undo-scopes-scope# entries must be deleted last, in the last deletion write batch, so this scope isn't mistaken later as something to be reverted.
+* Output - nothing
+
+# Lock Manager
+
+The scopes feature needs a fairly generic lock manager. This lock manager needs two levels, because versionchange transactions need to lock a whole database, where other transactions will lock smaller ranges within the level one range.
+
+### API Methods
+
+#### AcquireLock
+
+Accepts a lock type (shared or exclusive), a level number, a key range, and a callback which is given a moveable-only lock object, which releases its lock on destruction. The levels are totally independent from the perspective of the lock manager. IF an application wants to use multiple levels, they should employ an acquisition order (like, always acquire locks in increasing level order) so they don't deadlock.
+
+### Internal Data Structure
+
+The lock manager basically holds ranges, and needs to know if a new range intersects any old ranges. A good data structure for this is an Interval Tree.
+
+# Undo Operations
+
+## Types
+
+* `Put(key, value)`
+* `Delete(key)`
+* `DeleteRange(key_range)`
+
+## Generation
+
+### Normal Cases
+
+Note - all cases where "old_value" is used requires reading the current value from the database.
+
+**`Put(key, value)`**
+
+* `Delete(key)` if an old value doesn't exist,
+* `Put(key, old_value)` if an old value does exist, or
+* Nothing if the old value and new value matches
+
+**`Add(key, value)`**
+
+* Delete(key), trusting the client that there wasn't a value here before.
+
+**`Delete(key)`**
+
+* `Put(key, old_value)` if the old_value exists, or
+* Nothing if no old_value exists.
+
+**`DeleteRange(key_range)`**
+
+* `Put(key, old_value)` for every entry in that key range. This requires iterating the database using the key_range to find all entries.
+
+### Special Case - Empty Unique Key Range
+
+#### Creation - key range is empty initially
+
+If the values being created are in a key range that is initially empty (we trust the API caller here - there is no verification), and if the key range will never be reused if it is reverted, then the undo log can consist of a single:
+
+`DeleteRange(key_range)`
+
+Examples of this are creating new databases or indexes in a versionchange transaction. The scopes system can check to make sure it's range is empty before doing operations. This can either be done as a hint (per-range, per-scope), or always.
+
+#### Deletion - key range will never be used again.
+
+This is done by having commit ranges in the value of the commit point. A new scope is created, and commited, with the key ranges never-to-be-used-again will be the value of the commit point record.

content/browser/indexed_db/scopes/disjoint_range_lock_manager.cc

+// Copyright 2018 The Chromium Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file.
+
+#include "content/browser/indexed_db/scopes/disjoint_range_lock_manager.h"
+
+namespace content {
+
+DisjointRangeLockManager::LockRequest::LockRequest() = default;
+DisjointRangeLockManager::LockRequest::LockRequest(LockType type,
+                                                   LockAquiredCallback callback)
+    : requested_type(type), callback(std::move(callback)) {}
+DisjointRangeLockManager::LockRequest::LockRequest(LockRequest&&) = default;
+DisjointRangeLockManager::LockRequest::~LockRequest() = default;
+DisjointRangeLockManager::Lock::Lock() = default;
+DisjointRangeLockManager::Lock::Lock(Lock&&) = default;
+DisjointRangeLockManager::Lock::~Lock() = default;
+DisjointRangeLockManager::Lock& DisjointRangeLockManager::Lock::operator=(
+    DisjointRangeLockManager::Lock&&) = default;
+
+DisjointRangeLockManager::DisjointRangeLockManager(
+    int level_count,
+    const leveldb::Comparator* comparator,
+    scoped_refptr<base::SequencedTaskRunner> task_runner)
+    : comparator_(comparator), task_runner_(task_runner), weak_factory_(this) {
+  for (int level = 0; level < level_count; ++level) {
+    locks_.emplace_back(RangeLessThan(comparator));
+  }
+}
+
+DisjointRangeLockManager::~DisjointRangeLockManager() = default;
+
+int64_t DisjointRangeLockManager::LocksHeldForTesting() const {
+  DCHECK_CALLED_ON_VALID_SEQUENCE(sequence_checker_);
+  int64_t locks = 0;
+  for (const LockLevelMap& map : locks_) {
+    for (const auto& pair : map) {
+      locks += pair.second.acquired_count;
+    }
+  }
+  return locks;
+}
+int64_t DisjointRangeLockManager::RequestsWaitingForTesting() const {
+  DCHECK_CALLED_ON_VALID_SEQUENCE(sequence_checker_);
+  int64_t requests = 0;
+  for (const LockLevelMap& map : locks_) {
+    for (const auto& pair : map) {
+      requests += pair.second.queue.size();
+    }
+  }
+  return requests;
+}
+
+void DisjointRangeLockManager::AcquireLock(int level,
+                                           const LockRange& range,
+                                           LockType type,
+                                           LockAquiredCallback callback) {
+  DCHECK_CALLED_ON_VALID_SEQUENCE(sequence_checker_);
+  DCHECK_LT(level, static_cast<int>(locks_.size()));
+  DCHECK_LT(comparator_->Compare(leveldb::Slice(range.begin),
+                                 leveldb::Slice(range.end)),
+            0)
+      << "Range is invalid: " << range;
+
+  auto& level_locks = locks_[level];
+
+  auto it = level_locks.find(range);
+  if (it == level_locks.end()) {
+    it = level_locks
+             .emplace(std::piecewise_construct, std::forward_as_tuple(range),
+                      std::forward_as_tuple())
+             .first;
+  }
+
+  DCHECK_EQ(it->first, range)
+      << range << " does not match what is in the lock map " << it->first;
+#if DCHECK_IS_ON()
+  DCHECK(IsRangeDisjointFromNeighbors(level_locks, range, comparator_))
+      << "Range not discrete: " << range;
+#endif
+  Lock& lock = it->second;
+
+  if (lock.CanBeAcquired(type)) {
+    ++lock.acquired_count;
+    lock.lock_mode = type;
+    std::move(callback).Run(
+        ScopeLock(range, level,
+                  base::BindOnce(&DisjointRangeLockManager::LockReleased,
+                                 weak_factory_.GetWeakPtr(), level, range)));
+  } else {
+    lock.queue.emplace_back(type, std::move(callback));
+  }
+}
+
+void DisjointRangeLockManager::RemoveLockRange(int level,
+                                               const LockRange& range) {
+  DCHECK_CALLED_ON_VALID_SEQUENCE(sequence_checker_);
+  DCHECK_LT(level, static_cast<int>(locks_.size()));
+  auto& level_locks = locks_[level];
+  auto it = level_locks.find(range);
+  if (it != level_locks.end()) {
+    DCHECK_EQ(0, it->second.acquired_count);
+    level_locks.erase(it);
+  }
+}
+
+void DisjointRangeLockManager::LockReleased(int level, LockRange range) {
+  DCHECK_CALLED_ON_VALID_SEQUENCE(sequence_checker_);
+  DCHECK_LT(level, static_cast<int>(locks_.size()));
+  auto& level_locks = locks_[level];
+  auto it = level_locks.find(range);
+  DCHECK(it != level_locks.end());
+  Lock& lock = it->second;
+
+  DCHECK_GT(lock.acquired_count, 0);
+  --(lock.acquired_count);
+  if (lock.acquired_count == 0) {
+    // Either the lock isn't acquired yet or more shared locks can be granted.
+    while (!lock.queue.empty() &&
+           (lock.acquired_count == 0 ||
+            lock.queue.front().requested_type == LockType::kShared)) {
+      LockRequest requester = std::move(lock.queue.front());
+      lock.queue.pop_front();
+      ++lock.acquired_count;
+      lock.lock_mode = requester.requested_type;
+      task_runner_->PostTask(
+          FROM_HERE,
+          base::BindOnce(
+              std::move(requester.callback),
+              ScopeLock(range, level,
+                        base::BindOnce(&DisjointRangeLockManager::LockReleased,
+                                       weak_factory_.GetWeakPtr(), level,
+                                       std::move(range)))));
+      // This can only happen if acquired_count was 0.
+      if (requester.requested_type == LockType::kExclusive)
+        return;
+    }
+  }
+}
+
+#if DCHECK_IS_ON()
+// static
+bool DisjointRangeLockManager::IsRangeDisjointFromNeighbors(
+    const LockLevelMap& map,
+    const LockRange& range,
+    const leveldb::Comparator* const comparator) {
+  DCHECK_EQ(map.count(range), 1ull);
+  auto it = map.find(range);
+  auto next_it = it;
+  ++next_it;
+  if (next_it != map.end()) {
+    return comparator->Compare(leveldb::Slice(range.end),
+                               leveldb::Slice(next_it->first.begin)) <= 0;
+  }
+  auto last_it = it;
+  if (last_it != map.begin()) {
+    --last_it;
+    return comparator->Compare(leveldb::Slice(last_it->first.end),
+                               leveldb::Slice(range.begin)) <= 0;
+  }
+  return true;
+}
+#endif
+
+}  // namespace content

content/browser/indexed_db/scopes/disjoint_range_lock_manager.h

+// Copyright 2018 The Chromium Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file.
+
+#ifndef CONTENT_BROWSER_INDEXED_DB_SCOPES_DISJOINT_RANGE_LOCK_MANAGER_H_
+#define CONTENT_BROWSER_INDEXED_DB_SCOPES_DISJOINT_RANGE_LOCK_MANAGER_H_
+
+#include <list>
+#include <vector>
+
+#include "base/containers/flat_map.h"
+#include "base/memory/ref_counted.h"
+#include "base/memory/weak_ptr.h"
+#include "base/optional.h"
+#include "base/sequence_checker.h"
+#include "base/sequenced_task_runner.h"
+#include "content/browser/indexed_db/scopes/scopes_lock_manager.h"
+#include "content/common/content_export.h"
+#include "third_party/leveldatabase/src/include/leveldb/comparator.h"
+#include "third_party/leveldatabase/src/include/leveldb/slice.h"
+
+namespace content {
+
+// Holds locks of the scopes system. To be performant without an Interval Tree,
+// this implementation has the following invariants:
+// * All lock range requests at a level must be disjoint - they cannot overlap.
+// * Lock ranges are remembered for future performance - remove them using
+//   RemoveLockRange.
+// * All calls must happen from the same sequenced task runner.
+class CONTENT_EXPORT DisjointRangeLockManager : public ScopesLockManager {
+ public:
+  // Creates a lock manager with the given number of levels, the comparator for
+  // leveldb keys, and the current task runner that we are running on. The task
+  // runner will be used for the lock acquisition callbacks.
+  DisjointRangeLockManager(
+      int level_count,
+      const leveldb::Comparator* comparator,
+      scoped_refptr<base::SequencedTaskRunner> task_runner);
+  ~DisjointRangeLockManager() override;
+
+  int64_t LocksHeldForTesting() const override;
+  int64_t RequestsWaitingForTesting() const override;
+
+  void AcquireLock(int level,
+                   const LockRange& range,
+                   LockType type,
+                   LockAquiredCallback callback) override;
+
+  // Remove the given lock range at the given level. The lock range must not be
+  // in use. Use this if the lock will never be used again.
+  void RemoveLockRange(int level, const LockRange& range);
+
+ private:
+  struct LockRequest {
+   public:
+    LockRequest();
+    LockRequest(LockRequest&&) noexcept;
+    LockRequest(LockType type, LockAquiredCallback callback);
+    ~LockRequest();
+
+    LockType requested_type = LockType::kShared;
+    LockAquiredCallback callback;
+  };
+
+  // Represents a lock, which has a range and a level. To support shared access,
+  // there can be multiple acquisitions of this lock, represented in
+  // |acquired_count|. Also holds the pending requests for this lock.
+  struct Lock {
+   public:
+    Lock();
+    Lock(Lock&&) noexcept;
+    ~Lock();
+    Lock& operator=(Lock&&) noexcept;
+
+    bool CanBeAcquired(LockType lock_type) {
+      return acquired_count == 0 ||
+             (queue.empty() && this->lock_mode == LockType::kShared &&
+              lock_type == LockType::kShared);
+    }
+
+    int acquired_count = 0;
+    LockType lock_mode = LockType::kShared;
+    std::list<LockRequest> queue;
+
+   private:
+    DISALLOW_COPY_AND_ASSIGN(Lock);
+  };
+
+  // This is a proxy between a LevelDB Comparator and the interface expected by
+  // std::map. It sorts using the |begin| entry of the ranges.
+  class RangeLessThan {
+   public:
+    explicit RangeLessThan(const leveldb::Comparator* comparator)
+        : comparator_(comparator) {}
+    bool operator()(const LockRange& a, const LockRange& b) const {
+      return comparator_->Compare(leveldb::Slice(a.begin),
+                                  leveldb::Slice(b.begin)) < 0;
+    }
+
+   private:
+    const leveldb::Comparator* const comparator_;
+  };
+
+  using LockLevelMap = base::flat_map<LockRange, Lock, RangeLessThan>;
+
+  void LockReleased(int level, LockRange range);
+
+#if DCHECK_IS_ON()
+  static bool IsRangeDisjointFromNeighbors(
+      const LockLevelMap& map,
+      const LockRange& range,
+      const leveldb::Comparator* const comparator_);
+#endif
+
+  const leveldb::Comparator* const comparator_;
+  const scoped_refptr<base::SequencedTaskRunner> task_runner_;
+  // This vector should never be modified after construction.
+  std::vector<LockLevelMap> locks_;
+
+  SEQUENCE_CHECKER(sequence_checker_);
+  base::WeakPtrFactory<DisjointRangeLockManager> weak_factory_;
+  DISALLOW_COPY_AND_ASSIGN(DisjointRangeLockManager);
+};
+
+}  // namespace content
+
+#endif  // CONTENT_BROWSER_INDEXED_DB_SCOPES_DISJOINT_RANGE_LOCK_MANAGER_H_

content/browser/indexed_db/scopes/disjoint_range_lock_manager_unittest.cc

+// Copyright 2018 The Chromium Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file.
+
+#include <content/browser/indexed_db/scopes/disjoint_range_lock_manager.h>
+#include <content/test/barrier_builder.h>
+
+#include "base/barrier_closure.h"
+#include "base/bind_helpers.h"
+#include "base/run_loop.h"
+#include "base/strings/stringprintf.h"
+#include "base/test/bind_test_util.h"
+#include "base/test/scoped_task_environment.h"
+#include "base/threading/sequenced_task_runner_handle.h"
+#include "testing/gtest/include/gtest/gtest.h"
+
+namespace content {
+namespace {
+using ScopeLock = ScopesLockManager::ScopeLock;
+using LockRange = ScopesLockManager::LockRange;
+
+template <typename T>
+void SetValue(T* out, T value) {
+  *out = value;
+}
+
+std::string IntegerKey(size_t num) {
+  return base::StringPrintf("%010zd", num);
+}
+
+void StoreLock(ScopeLock* lock_out, base::OnceClosure done, ScopeLock lock) {
+  (*lock_out) = std::move(lock);
+  std::move(done).Run();
+}
+
+class DisjointRangeLockManagerTest : public testing::Test {
+ public:
+  DisjointRangeLockManagerTest() = default;
+  ~DisjointRangeLockManagerTest() override = default;
+
+ private:
+  base::test::ScopedTaskEnvironment task_env_;
+};
+
+TEST_F(DisjointRangeLockManagerTest, BasicAcquisition) {
+  const size_t kTotalLocks = 1000;
+  DisjointRangeLockManager lock_manager(1, leveldb::BytewiseComparator(),
+                                        base::SequencedTaskRunnerHandle::Get());
+
+  EXPECT_EQ(0ll, lock_manager.LocksHeldForTesting());
+  EXPECT_EQ(0ll, lock_manager.RequestsWaitingForTesting());
+
+  base::RunLoop loop;
+  std::vector<ScopeLock> locks;
+  locks.resize(kTotalLocks);
+  {
+    BarrierBuilder barrier(loop.QuitClosure());
+
+    for (size_t i = 0; i < kTotalLocks / 2; ++i) {
+      LockRange range = {IntegerKey(i), IntegerKey(i + 1)};
+      lock_manager.AcquireLock(
+          0, range, ScopesLockManager::LockType::kExclusive,
+          base::BindOnce(&StoreLock, &locks[i], barrier.AddClosure()));
+    }
+
+    for (size_t i = kTotalLocks - 1; i >= kTotalLocks / 2; --i) {
+      LockRange range = {IntegerKey(i), IntegerKey(i + 1)};
+      lock_manager.AcquireLock(
+          0, range, ScopesLockManager::LockType::kExclusive,
+          base::BindOnce(&StoreLock, &locks[i], barrier.AddClosure()));
+    }
+  }
+  loop.Run();
+  EXPECT_EQ(static_cast<int64_t>(kTotalLocks),
+            lock_manager.LocksHeldForTesting());
+  EXPECT_EQ(0ll, lock_manager.RequestsWaitingForTesting());
+  // All locks should be acquired.
+  for (const auto& lock : locks) {
+    EXPECT_TRUE(lock.is_locked());
+  }
+
+  locks.clear();
+  EXPECT_EQ(0ll, lock_manager.LocksHeldForTesting());
+}
+
+TEST_F(DisjointRangeLockManagerTest, Shared) {
+  DisjointRangeLockManager lock_manager(1, leveldb::BytewiseComparator(),
+                                        base::SequencedTaskRunnerHandle::Get());
+  EXPECT_EQ(0ll, lock_manager.LocksHeldForTesting());
+  EXPECT_EQ(0ll, lock_manager.RequestsWaitingForTesting());
+
+  LockRange range = {IntegerKey(0), IntegerKey(1)};
+
+  ScopeLock lock1;
+  ScopeLock lock2;
+  base::RunLoop loop;
+  {
+    BarrierBuilder barrier(loop.QuitClosure());
+    lock_manager.AcquireLock(
+        0, range, ScopesLockManager::LockType::kShared,
+        base::BindOnce(&StoreLock, &lock1, barrier.AddClosure()));
+    lock_manager.AcquireLock(
+        0, range, ScopesLockManager::LockType::kShared,
+        base::BindOnce(&StoreLock, &lock2, barrier.AddClosure()));
+  }
+  loop.Run();
+  EXPECT_EQ(2ll, lock_manager.LocksHeldForTesting());
+
+  EXPECT_TRUE(lock1.is_locked());
+  EXPECT_TRUE(lock2.is_locked());
+}
+
+TEST_F(DisjointRangeLockManagerTest, SharedAndExclusiveQueuing) {
+  DisjointRangeLockManager lock_manager(1, leveldb::BytewiseComparator(),
+                                        base::SequencedTaskRunnerHandle::Get());
+  EXPECT_EQ(0ll, lock_manager.LocksHeldForTesting());
+  EXPECT_EQ(0ll, lock_manager.RequestsWaitingForTesting());
+
+  LockRange range = {IntegerKey(0), IntegerKey(1)};
+
+  ScopeLock shared_lock1;
+  ScopeLock shared_lock2;
+  ScopeLock exclusive_lock3;
+  ScopeLock shared_lock4;
+
+  {
+    base::RunLoop loop;
+    base::RepeatingClosure barrier =
+        base::BarrierClosure(2, loop.QuitClosure());
+    lock_manager.AcquireLock(
+        0, range, ScopesLockManager::LockType::kShared,
+        base::BindOnce(&StoreLock, &shared_lock1, barrier));
+    lock_manager.AcquireLock(
+        0, range, ScopesLockManager::LockType::kShared,
+        base::BindOnce(&StoreLock, &shared_lock2, barrier));
+    loop.Run();
+  }
+
+  // Both of the following locks should be queued - the exclusive is next in
+  // line, then the shared lock will come after it.
+  lock_manager.AcquireLock(
+      0, range, ScopesLockManager::LockType::kExclusive,
+      base::BindOnce(&StoreLock, &exclusive_lock3, base::DoNothing::Once()));
+  lock_manager.AcquireLock(
+      0, range, ScopesLockManager::LockType::kShared,
+      base::BindOnce(&StoreLock, &shared_lock4, base::DoNothing::Once()));
+  // Flush the task queue.
+  {
+    base::RunLoop loop;
+    base::SequencedTaskRunnerHandle::Get()->PostTask(FROM_HERE,
+                                                     loop.QuitClosure());
+    loop.Run();
+  }
+  EXPECT_FALSE(exclusive_lock3.is_locked());
+  EXPECT_FALSE(shared_lock4.is_locked());
+  EXPECT_EQ(2ll, lock_manager.LocksHeldForTesting());
+  EXPECT_EQ(2ll, lock_manager.RequestsWaitingForTesting());
+
+  // Release the shared locks.
+  shared_lock1.Release();
+  shared_lock2.Release();
+  EXPECT_FALSE(shared_lock1.is_locked());
+  EXPECT_FALSE(shared_lock2.is_locked());
+
+  // Flush the task queue to propagate the lock releases and grant the exclusive
+  // lock.
+  {
+    base::RunLoop loop;
+    base::SequencedTaskRunnerHandle::Get()->PostTask(FROM_HERE,
+                                                     loop.QuitClosure());
+    loop.Run();
+  }
+  EXPECT_TRUE(exclusive_lock3.is_locked());
+  EXPECT_FALSE(shared_lock4.is_locked());
+  EXPECT_EQ(1ll, lock_manager.LocksHeldForTesting());
+  EXPECT_EQ(1ll, lock_manager.RequestsWaitingForTesting());
+
+  exclusive_lock3.Release();
+  EXPECT_FALSE(exclusive_lock3.is_locked());
+
+  // Flush the task queue to propagate the lock releases and grant the exclusive
+  // lock.
+  {
+    base::RunLoop loop;
+    base::SequencedTaskRunnerHandle::Get()->PostTask(FROM_HERE,
+                                                     loop.QuitClosure());
+    loop.Run();
+  }
+  EXPECT_TRUE(shared_lock4.is_locked());
+  EXPECT_EQ(1ll, lock_manager.LocksHeldForTesting());
+  EXPECT_EQ(0ll, lock_manager.RequestsWaitingForTesting());
+}
+
+TEST_F(DisjointRangeLockManagerTest, LevelsOperateSeparately) {
+  DisjointRangeLockManager lock_manager(2, leveldb::BytewiseComparator(),
+                                        base::SequencedTaskRunnerHandle::Get());
+  base::RunLoop loop;
+  ScopeLock l0_lock;
+  ScopeLock l1_lock;
+  {
+    BarrierBuilder barrier(loop.QuitClosure());
+    LockRange range = {IntegerKey(0), IntegerKey(1)};
+    lock_manager.AcquireLock(
+        0, range, ScopesLockManager::LockType::kExclusive,
+        base::BindOnce(&StoreLock, &l0_lock, barrier.AddClosure()));
+    lock_manager.AcquireLock(
+        1, range, ScopesLockManager::LockType::kExclusive,
+        base::BindOnce(&StoreLock, &l1_lock, barrier.AddClosure()));
+  }
+  loop.Run();
+  EXPECT_TRUE(l0_lock.is_locked());
+  EXPECT_TRUE(l1_lock.is_locked());
+  EXPECT_EQ(2ll, lock_manager.LocksHeldForTesting());
+  EXPECT_EQ(0ll, lock_manager.RequestsWaitingForTesting());
+  l0_lock.Release();
+  l1_lock.Release();
+  EXPECT_EQ(0ll, lock_manager.LocksHeldForTesting());
+}
+
+}  // namespace
+}  // namespace content

content/browser/indexed_db/scopes/scopes_lock_manager.cc

+// Copyright 2018 The Chromium Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file.
+
+#include "content/browser/indexed_db/scopes/scopes_lock_manager.h"
+
+#include <ostream>
+
+namespace content {
+
+ScopesLockManager::LockRange::LockRange(std::string begin, std::string end)
+    : begin(std::move(begin)), end(std::move(end)) {}
+
+ScopesLockManager::ScopeLock::ScopeLock() = default;
+ScopesLockManager::ScopeLock::ScopeLock(ScopeLock&& other) {
+  DCHECK(!this->is_locked_) << "Cannot move a lock onto an active lock.";
+  this->is_locked_ = other.is_locked_;
+  this->range_ = std::move(other.range_);
+  this->level_ = other.level_;
+  this->closure_runner_ = std::move(other.closure_runner_);
+  other.is_locked_ = false;
+}
+ScopesLockManager::ScopeLock::ScopeLock(LockRange range,
+                                        int level,
+                                        base::OnceClosure closure)
+    : is_locked_(!closure.is_null()),
+      range_(std::move(range)),
+      level_(level),
+      closure_runner_(std::move(closure)) {}
+
+ScopesLockManager::ScopeLock& ScopesLockManager::ScopeLock::operator=(
+    ScopesLockManager::ScopeLock&& other) {
+  DCHECK(!this->is_locked_);
+  this->is_locked_ = other.is_locked_;
+  this->range_ = std::move(other.range_);
+  this->level_ = other.level_;
+  this->closure_runner_ = std::move(other.closure_runner_);
+  other.is_locked_ = false;
+  return *this;
+};
+
+void ScopesLockManager::ScopeLock::Release() {
+  is_locked_ = false;
+  closure_runner_.RunAndReset();
+}
+
+std::ostream& operator<<(std::ostream& out,
+                         const ScopesLockManager::LockRange& range) {
+  return out << "<ScopesLockManager::ScopeLock>{begin: " << range.begin
+             << ", end: " << range.end << "}";
+}
+
+bool operator==(const ScopesLockManager::LockRange& x,
+                const ScopesLockManager::LockRange& y) {
+  return x.begin == y.begin && x.end == y.end;
+}
+bool operator!=(const ScopesLockManager::LockRange& x,
+                const ScopesLockManager::LockRange& y) {
+  return !(x == y);
+}
+}  // namespace content

content/browser/indexed_db/scopes/scopes_lock_manager.h

+// Copyright 2018 The Chromium Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file.
+
+#ifndef CONTENT_BROWSER_INDEXED_DB_SCOPES_SCOPES_LOCK_MANAGER_H_
+#define CONTENT_BROWSER_INDEXED_DB_SCOPES_SCOPES_LOCK_MANAGER_H_
+
+#include <iosfwd>
+#include <string>
+
+#include "base/callback.h"
+#include "base/callback_helpers.h"
+#include "base/logging.h"
+#include "base/macros.h"
+#include "content/common/content_export.h"
+
+namespace content {
+
+// Generic two-level lock management system based on ranges. Granted locks are
+// represented by the |ScopeLock| class.
+class CONTENT_EXPORT ScopesLockManager {
+ public:
+  // Shared locks can share access to a lock range, while exclusive locks
+  // require that they are the only lock for their range.
+  enum class LockType { kShared, kExclusive };
+
+  // The range is [begin, end).
+  struct CONTENT_EXPORT LockRange {
+    LockRange(std::string begin, std::string end);
+    LockRange() = default;
+    ~LockRange() = default;
+    std::string begin;
+    std::string end;
+  };
+
+  // Represents a granted lock in the ScopesLockManager. When this object is
+  // destroyed, the lock is released. Since default construction is supported,
+  // |is_locked()| can be used to inquire locked status. Also, |Release()| can
+  // be called to manually release the lock, which appropriately updates the
+  // |is_locked()| result.
+  class CONTENT_EXPORT ScopeLock {
+   public:
+    ScopeLock();
+    ScopeLock(ScopeLock&&) noexcept;
+    // The |closure| is called when the lock is released, either by destruction
+    // of this object or by the |Released()| call. It will be called
+    // synchronously on the sequence runner this lock is released on.
+    ScopeLock(LockRange range, int level, base::OnceClosure closure);
+    ~ScopeLock() = default;
+    // This does NOT release the lock if one is being held.
+    ScopeLock& operator=(ScopeLock&&) noexcept;
+
+    // Returns true if this object is holding a lock.
+    bool is_locked() const { return is_locked_; }
+
+    // Releases this lock.
+    void Release();
+
+    int level() const { return level_; }
+    const LockRange& range() const { return range_; }
+
+   private:
+    bool is_locked_ = false;
+    LockRange range_;
+    int level_ = 0;
+    base::ScopedClosureRunner closure_runner_;
+
+    DISALLOW_COPY_AND_ASSIGN(ScopeLock);
+  };
+
+  using LockAquiredCallback = base::OnceCallback<void(ScopeLock)>;
+
+  ScopesLockManager() = default;
+
+  virtual ~ScopesLockManager() = default;
+
+  virtual int64_t LocksHeldForTesting() const = 0;
+  virtual int64_t RequestsWaitingForTesting() const = 0;
+
+  // Acquires a lock for a given lock level. Lock levels are treated as
+  // completely independent domains. The lock levels start at zero.
+  virtual void AcquireLock(int level,
+                           const LockRange& range,
+                           LockType type,
+                           LockAquiredCallback callback) = 0;
+
+ private:
+  DISALLOW_COPY_AND_ASSIGN(ScopesLockManager);
+};
+
+// Stream operator so lock range can be used in log statements.
+CONTENT_EXPORT std::ostream& operator<<(
+    std::ostream& out,
+    const ScopesLockManager::LockRange& range);
+
+CONTENT_EXPORT bool operator==(const ScopesLockManager::LockRange& x,
+                               const ScopesLockManager::LockRange& y);
+CONTENT_EXPORT bool operator!=(const ScopesLockManager::LockRange& x,
+                               const ScopesLockManager::LockRange& y);
+
+}  // namespace content
+
+#endif  // CONTENT_BROWSER_INDEXED_DB_SCOPES_SCOPES_LOCK_MANAGER_H_

content/browser/indexed_db/scopes/scopes_lock_manager_unittest.cc

+// Copyright 2018 The Chromium Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file.
+
+#include "content/browser/indexed_db/scopes/scopes_lock_manager.h"
+
+#include "testing/gtest/include/gtest/gtest.h"
+
+namespace content {
+
+TEST(ScopesLockManager, TestRangePopulation) {
+  ScopesLockManager::LockRange range("a", "b");
+  EXPECT_EQ("a", range.begin);
+  EXPECT_EQ("b", range.end);
+}
+
+}  // namespace content

content/test/BUILD.gn

@@ -196,6 +196,8 @@ jumbo_static_library("test_support") {
     "accessibility_browser_test_utils.h",
     "appcache_test_helper.cc",
     "appcache_test_helper.h",
+    "barrier_builder.cc",
+    "barrier_builder.h",
     "content_browser_sanity_checker.cc",
     "content_browser_sanity_checker.h",
     "content_test_suite.cc",
@@ -1399,6 +1401,8 @@ test("content_unittests") {
     "../browser/indexed_db/mock_mojo_indexed_db_callbacks.h",
     "../browser/indexed_db/mock_mojo_indexed_db_database_callbacks.cc",
     "../browser/indexed_db/mock_mojo_indexed_db_database_callbacks.h",
+    "../browser/indexed_db/scopes/disjoint_range_lock_manager_unittest.cc",
+    "../browser/indexed_db/scopes/scopes_lock_manager_unittest.cc",
     "../browser/loader/cross_site_document_resource_handler_unittest.cc",
     "../browser/loader/data_pipe_to_source_stream_unittest.cc",
     "../browser/loader/detachable_resource_handler_unittest.cc",

content/test/barrier_builder.cc

+// Copyright 2018 The Chromium Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file.
+
+#include "content/test/barrier_builder.h"
+
+#include "base/bind.h"
+#include "base/callback_helpers.h"
+
+namespace content {
+
+// The callbacks returned by |AddClosure| can get called after destruction of
+// BarrierBuilder, so there needs to be an internal class to hold the final
+// callback.
+class BarrierBuilder::InternalBarrierBuilder
+    : public base::RefCountedThreadSafe<
+          BarrierBuilder::InternalBarrierBuilder> {
+ public:
+  InternalBarrierBuilder(base::OnceClosure done_closure)
+      : done_runner_(std::move(done_closure)) {}
+
+ private:
+  friend class base::RefCountedThreadSafe<
+      BarrierBuilder::InternalBarrierBuilder>;
+
+  ~InternalBarrierBuilder() = default;
+
+  base::ScopedClosureRunner done_runner_;
+
+  DISALLOW_COPY_AND_ASSIGN(InternalBarrierBuilder);
+};
+
+BarrierBuilder::BarrierBuilder(base::OnceClosure done_closure)
+    : internal_barrier_(
+          new BarrierBuilder::InternalBarrierBuilder(std::move(done_closure))) {
+}
+
+BarrierBuilder::~BarrierBuilder() = default;
+
+base::OnceClosure BarrierBuilder::AddClosure() {
+  return base::BindOnce(
+      [](scoped_refptr<BarrierBuilder::InternalBarrierBuilder>) {},
+      internal_barrier_);
+}
+
+}  // namespace content

content/test/barrier_builder.h

+// Copyright 2018 The Chromium Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file.
+
+#ifndef CONTENT_TEST_BARRIER_BUILDER_H_
+#define CONTENT_TEST_BARRIER_BUILDER_H_
+
+#include "base/callback.h"
+#include "base/macros.h"
+#include "base/memory/ref_counted.h"
+
+namespace content {
+
+// A BarrierBuilder helps create a barrier for a large, complex, or variable
+// number of steps.
+//
+// Example usage:
+//
+// base::RunLoop loop;
+// {
+//   BarrierBuilder barrier(loop.QuitClosure());
+//   AsyncMethod(barrier.AddClosure());
+//   AsyncMethod(barrier.AddClosure());
+// }
+// loop.Run();
+//
+// Once all closures returned by AddClosure AND the BarrierBuilder object are
+// all destroyed, the |done_closure| is called. The |done_closure| will
+// be invoked on the thread that calls the last step closure, or on the thread
+// where BarrierBuilder goes out of scope (if that happens after all closures
+// are called).
+class BarrierBuilder {
+ public:
+  explicit BarrierBuilder(base::OnceClosure done_closure);
+  ~BarrierBuilder();
+
+  // Returns a closure that can be used in the same way that a barrier closure
+  // would be used. Furthermore, each callback returned by AddClosure() should
+  // eventually be run, or else 'done_closure' will never be run.
+  base::OnceClosure AddClosure();
+
+ private:
+  class InternalBarrierBuilder;
+
+  scoped_refptr<InternalBarrierBuilder> internal_barrier_;
+
+  DISALLOW_COPY_AND_ASSIGN(BarrierBuilder);
+};
+
+}  // namespace content
+
+#endif  // CONTENT_TEST_BARRIER_BUILDER_H_