Commit 92544dcf authored by Nick Diego Yamane's avatar Nick Diego Yamane Committed by Commit Bot

exo: Add extended-drag extension and its stub implementation

TL;DR:

Wayland Protocol needs to be extended to make it possible to properly
support full Chromium's tab dragging experience. Further details in the
Design document [1]. This is the first of a patch series which
implements extended-drag extension in Exo compositor, adding the
protocol descriptor XML file, boilerplate code as well as a stub
implementation of it. Follow-up CLs will incrementally implement the
extension features and start using it in Chromium Ozone/Wayland
(client-side).

-----------------------------------------------------------------------
An overview of how this protocol is aimed to be used in a tabdrag-like
scenario, but in generic terms, is outlined below:

1. An UI item (eg: a tab in a tab bar) starts to be dragged within shell
surface A owned by source client S. At some point, it gets far enough
from its original location and S then starts a Wayland "extended drag"
session, which protocol-wise requires to:

1.1. Create a wl_data_source, set one (or more) mime type and the
supported dnd actions;

1.2. Create a zcr_extended_drag_source for the newly created drag
source. Here it is possible to set a few options, such as:
  - ALLOW_SWALLOW: determines that the dragged item can be attached (or
  incorporated, also referred to as "swallow" in this protocol) to
  another surface, including use cases such as Chrome's tab drag.
  - ALLOW_DROP_NO_TARGET: In the standard DND protocol, dropping
  something outside shell surfaces lead to a wl_data_source::cancel,
  which is problematic in use cases such as tab drag, preventing us from
  detecting when it was really cancelled, e.g: ESC pressed.
  - LOCK_CURSOR: Keep the cursor shape unchanged during the whole
  session. Ultimately, having an extended drag source attached to the
  wl_data_source used to wl_data_device::start_drag instructs the
  compositor to run the drag session in the "extended mode".

1.3 Start the drag session as usual (i.e: wl_data_source::start_drag),
with a null drag icon surface. Assuming that we're still in attached
mode for now. In addition, from now on, client S must start watching for
wl_data_device::{enter,leave,motion} events in order to be able to
detect the next actions and state transitions as the user drags the UI
item around.

1.4 Store the (x_offset, y_offset) location where the drag started in S
and keep them synced to the values received in wl_data_device::motion()
events.

2. Once the drag gets far enough from the original UI item's container
(ie: tab bar), it's time to detach (or unswallow) it from surface A. To
achieve it, client S must:

2.1. Create a new wl_surface B + its corresponding shell surface, e.g: a
toplevel xdg surface where a new detached browser window will rendered
to in Chrome's tab drag use case;

2.2. Issue a zcr_extended_drag_source::drag(B, x_offset, y_offset)
request, which tells the compositor (in advance) that B must be set as
the extended-drag surface, using x and y offsets to translate it from
the current pointer location;

2.3. Map surface B (configure, attach buffer, etc) as usual. After this
B's shell surface should show up tied to the mouse pointer and the
compositor is supposed to take care of it, putting it in a special state
where, for example, it does not receive events, etc, acting just like a
standard DND "drag icon". At this point, the "unswallow" operation is
done and the extended-drag session enters in detached state.

3. Supposing the drop happens while in detached state, there are 2
possible flows, which depend on whether ALLOW_DROP_NO_TARGET config is
set or not (see 1.2). If it is set, wl_data_source::dnd_finished event
is sent to the source client S, otherwise wl_data_source::cancelled is
sent.

4. Otherwise, if before dropping, the pointer enters surface C, owned by
client T. T receives a wl_data_offer O through wl_data_device::offer
event as usual, to which it can attach a zcr_extended_drag_offer XO to
it by issuing zcr_extended_drag::get_extended_drag_offer(O). Target
client T, then starts monitoring wl_data_device::motion events, so that
it can trigger a swallow based on where the pointer is. If B is dragged
over a region that leads to a swallow, T must:

4.1 Issue a zcr_extended_drag_offer::swallow(serial, mime) request,
asking source client S to incorporate the dragged item into its UI.
S then receives a zcr_extended_drag_source::swallow(mime) event and
it can accept it by calling zcr_extended_drag_source::drag(null, 0, 0),
so client T can finally render it as part of its UI. At this point, the
session is back to a state similar to 1.4. So, similarly to 2, once the
drag gets far enough, T might want to unswallow (aka: detach) the UI
item. In order to do so it must:

4.1.1 Call unswallow(serial, mime, x_offset, y_offset) on the
extended-drag offer XO (created at step 4), whereas the offsets tell how
the dragged surface must be positioned related to the pointer location.
Client S will then receive a zcr_extended_drag_source::unswallow(mime,
x_offset, y_offset) event so that it can create and map a new surface D
and issue a zcr_extended_drag_source::drag(D, x_offset, y_offset), same
as in step 2.3, making surface D to show up under the pointer. At target
side, T can then re-render its UI without the dragged item. This
finishes the unswallow (or detaching) operation, transitioning the
session back to "detached" state.

[1] https://docs.google.com/document/d/1s6OwTi_WC-pS21WLGQYI39yw2m42ZlVolUXBclljXB4/edit?usp=sharing

Bug: 1099418
Change-Id: I8d876e96e45717c625c1d1bdcc269b100524cbc7
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2307653
Commit-Queue: Nick Yamane <nickdiego@igalia.com>
Reviewed-by: default avatarMitsuru Oshima <oshima@chromium.org>
Cr-Commit-Position: refs/heads/master@{#811055}
parent c913d715
......@@ -33,6 +33,10 @@ static_library("exo") {
"display.h",
"drag_drop_operation.cc",
"drag_drop_operation.h",
"extended_drag_offer.cc",
"extended_drag_offer.h",
"extended_drag_source.cc",
"extended_drag_source.h",
"frame_sink_resource_manager.cc",
"frame_sink_resource_manager.h",
"input_trace.h",
......
// Copyright 2020 The Chromium Authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.
#include "components/exo/extended_drag_offer.h"
#include <cstdint>
#include <string>
#include "base/check_op.h"
#include "base/notreached.h"
#include "components/exo/data_offer.h"
#include "components/exo/data_offer_observer.h"
#include "ui/gfx/geometry/vector2d.h"
namespace exo {
ExtendedDragOffer::ExtendedDragOffer(DataOffer* offer, Delegate* delegate)
: offer_(offer), delegate_(delegate) {
DCHECK(offer_);
DCHECK(delegate_);
}
ExtendedDragOffer::~ExtendedDragOffer() {
delegate_->OnDataOfferDestroying();
}
// TODO(crbug.com/1099418): Implement extended-drag Wayland extension.
void ExtendedDragOffer::Swallow(uint32_t serial, const std::string& mime_type) {
NOTIMPLEMENTED();
}
// TODO(crbug.com/1099418): Implement extended-drag Wayland extension.
void ExtendedDragOffer::Unswallow(uint32_t serial,
const std::string& mime_type,
const gfx::Vector2d& offset) {
NOTIMPLEMENTED();
}
void ExtendedDragOffer::OnDataOfferDestroying(DataOffer* offer) {
DCHECK_EQ(offer, offer_);
offer_ = nullptr;
}
} // namespace exo
// Copyright 2020 The Chromium Authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.
#ifndef COMPONENTS_EXO_EXTENDED_DRAG_OFFER_H_
#define COMPONENTS_EXO_EXTENDED_DRAG_OFFER_H_
#include <cstdint>
#include <string>
#include "components/exo/data_offer_observer.h"
namespace gfx {
class Vector2d;
}
namespace exo {
class DataOffer;
class ExtendedDragOffer : public DataOfferObserver {
public:
class Delegate {
public:
virtual void OnDataOfferDestroying() = 0;
protected:
virtual ~Delegate() = default;
};
ExtendedDragOffer(DataOffer* offer, Delegate* delegate);
ExtendedDragOffer(const ExtendedDragOffer&) = delete;
ExtendedDragOffer& operator=(const ExtendedDragOffer&) = delete;
~ExtendedDragOffer() override;
void Swallow(uint32_t serial, const std::string& mime_type);
void Unswallow(uint32_t serial,
const std::string& mime_type,
const gfx::Vector2d& offset);
private:
// DataOfferObserver:
void OnDataOfferDestroying(DataOffer* offer) override;
DataOffer* offer_ = nullptr;
// Created and destroyed at wayland/zcr_extended_drag.cc and its lifetime is
// tied to the zcr_extended_drag_source_v1 object it's attached to.
Delegate* const delegate_;
};
} // namespace exo
#endif // COMPONENTS_EXO_EXTENDED_DRAG_OFFER_H_
// Copyright 2020 The Chromium Authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.
#include "components/exo/extended_drag_source.h"
#include "base/check_op.h"
#include "base/logging.h"
#include "components/exo/data_source.h"
#include "components/exo/seat.h"
#include "components/exo/surface.h"
#include "ui/gfx/geometry/vector2d.h"
namespace exo {
ExtendedDragSource::ExtendedDragSource(DataSource* source,
Seat* seat,
Delegate* delegate)
: delegate_(delegate), seat_(seat), source_(source) {
DCHECK(source_);
DCHECK(seat_);
DCHECK(delegate_);
DVLOG(1) << "ExtendedDragSource created. wl_source=" << source_;
source_->AddObserver(this);
}
ExtendedDragSource::~ExtendedDragSource() {
delegate_->OnDataSourceDestroying();
for (auto& observer : observers_)
observer.OnExtendedDragSourceDestroying(this);
if (source_)
source_->RemoveObserver(this);
}
void ExtendedDragSource::AddObserver(Observer* observer) {
DCHECK(observer);
observers_.AddObserver(observer);
}
void ExtendedDragSource::RemoveObserver(Observer* observer) {
DCHECK(observer);
observers_.RemoveObserver(observer);
}
void ExtendedDragSource::Drag(Surface* dragged_surface,
const gfx::Vector2d& drag_offset) {
// Associated data source already destroyed.
if (!source_)
return;
if (dragged_surface == dragged_surface_ && drag_offset == drag_offset_)
return;
dragged_surface_ = dragged_surface;
drag_offset_ = drag_offset;
DVLOG(1) << "Dragged surface changed: surface=" << dragged_surface_
<< " offset=" << drag_offset_.ToString();
for (auto& observer : observers_)
observer.OnDraggedSurfaceChanged(this);
}
void ExtendedDragSource::OnDataSourceDestroying(DataSource* source) {
DCHECK_EQ(source, source_);
source_->RemoveObserver(this);
source_ = nullptr;
}
} // namespace exo
// Copyright 2020 The Chromium Authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.
#ifndef COMPONENTS_EXO_EXTENDED_DRAG_SOURCE_H_
#define COMPONENTS_EXO_EXTENDED_DRAG_SOURCE_H_
#include <string>
#include "base/observer_list.h"
#include "base/optional.h"
#include "components/exo/data_source_observer.h"
#include "ui/gfx/geometry/vector2d.h"
namespace exo {
class DataSource;
class Seat;
class Surface;
class ExtendedDragSource : public DataSourceObserver {
public:
class Delegate {
public:
virtual bool ShouldAllowDropAnywhere() const = 0;
virtual bool ShouldLockCursor() const = 0;
virtual void OnSwallowed(std::string mime_type) = 0;
virtual void OnUnswallowed(std::string mime_type,
const gfx::Vector2d& offset) = 0;
virtual void OnDataSourceDestroying() = 0;
protected:
virtual ~Delegate() = default;
};
class Observer {
public:
virtual void OnExtendedDragSourceDestroying(ExtendedDragSource* source) = 0;
virtual void OnDraggedSurfaceChanged(ExtendedDragSource* source) = 0;
protected:
virtual ~Observer() = default;
};
ExtendedDragSource(DataSource* source, Seat* seat, Delegate* delegate);
ExtendedDragSource(const ExtendedDragSource&) = delete;
ExtendedDragSource& operator=(const ExtendedDragSource&) = delete;
~ExtendedDragSource() override;
void AddObserver(Observer* observer);
void RemoveObserver(Observer* observer);
bool should_allow_drop_anywhere() const {
return delegate_->ShouldAllowDropAnywhere();
}
bool should_lock_cursor() const { return delegate_->ShouldLockCursor(); }
const gfx::Vector2d& drag_offset() const { return drag_offset_; }
void Drag(Surface* surface, const gfx::Vector2d& offset);
private:
// DataSourceObserver:
void OnDataSourceDestroying(DataSource* source) override;
// Created and destroyed at wayland/zcr_extended_drag.cc and its lifetime is
// tied to the zcr_extended_drag_source_v1 object it's attached to.
Delegate* const delegate_;
Seat* const seat_;
DataSource* source_ = nullptr;
Surface* dragged_surface_ = nullptr;
gfx::Vector2d drag_offset_;
base::ObserverList<Observer>::Unchecked observers_;
};
} // namespace exo
#endif // COMPONENTS_EXO_EXTENDED_DRAG_SOURCE_H_
......@@ -49,6 +49,8 @@ source_set("wayland") {
"zaura_shell.h",
"zcr_alpha_compositing.cc",
"zcr_alpha_compositing.h",
"zcr_extended_drag.cc",
"zcr_extended_drag.h",
"zcr_secure_output.cc",
"zcr_secure_output.h",
"zcr_stylus.cc",
......@@ -74,6 +76,7 @@ source_set("wayland") {
"//third_party/wayland-protocols:alpha_compositing_protocol",
"//third_party/wayland-protocols:color_space_protocol",
"//third_party/wayland-protocols:cursor_shapes_protocol",
"//third_party/wayland-protocols:extended_drag",
"//third_party/wayland-protocols:gaming_input_protocol",
"//third_party/wayland-protocols:input_timestamps_protocol",
"//third_party/wayland-protocols:keyboard_configuration_protocol",
......
......@@ -8,6 +8,7 @@
#include <aura-shell-server-protocol.h>
#include <color-space-unstable-v1-server-protocol.h>
#include <cursor-shapes-unstable-v1-server-protocol.h>
#include <extended-drag-unstable-v1-server-protocol.h>
#include <gaming-input-unstable-v2-server-protocol.h>
#include <grp.h>
#include <input-timestamps-unstable-v1-server-protocol.h>
......@@ -66,6 +67,7 @@
#include "components/exo/wayland/xdg_shell.h"
#include "components/exo/wayland/zcr_color_space.h"
#include "components/exo/wayland/zcr_cursor_shapes.h"
#include "components/exo/wayland/zcr_extended_drag.h"
#include "components/exo/wayland/zcr_gaming_input.h"
#include "components/exo/wayland/zcr_keyboard_configuration.h"
#include "components/exo/wayland/zcr_keyboard_extension.h"
......@@ -210,6 +212,8 @@ Server::Server(Display* display)
display_, bind_color_space);
wl_global_create(wl_display_.get(), &zxdg_decoration_manager_v1_interface, 1,
display_, bind_zxdg_decoration_manager);
wl_global_create(wl_display_.get(), &zcr_extended_drag_v1_interface, 1,
display_, bind_extended_drag);
zwp_text_manager_data_ = std::make_unique<WaylandTextInputManager>(
display_->seat()->xkb_tracker(), serial_tracker_.get());
......
// Copyright 2020 The Chromium Authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.
#include "components/exo/wayland/zcr_extended_drag.h"
#include <extended-drag-unstable-v1-server-protocol.h>
#include <wayland-server-core.h>
#include <wayland-server-protocol-core.h>
#include <cstdint>
#include "base/notreached.h"
#include "components/exo/data_offer.h"
#include "components/exo/data_source.h"
#include "components/exo/display.h"
#include "components/exo/extended_drag_offer.h"
#include "components/exo/extended_drag_source.h"
#include "components/exo/surface.h"
#include "components/exo/wayland/server_util.h"
#include "ui/gfx/geometry/vector2d.h"
namespace exo {
namespace wayland {
namespace {
////////////////////////////////////////////////////////////////////////////////
// zcr_extended_drag_source interface:
class ZcrExtendedDragSourceDelegate : public ExtendedDragSource::Delegate {
public:
ZcrExtendedDragSourceDelegate(wl_resource* resource, uint32_t settings)
: resource_(resource), settings_(settings) {}
ZcrExtendedDragSourceDelegate(const ZcrExtendedDragSourceDelegate&) = delete;
ZcrExtendedDragSourceDelegate& operator=(
const ZcrExtendedDragSourceDelegate&) = delete;
~ZcrExtendedDragSourceDelegate() override = default;
// ExtendedDragSource::Delegate:
bool ShouldAllowDropAnywhere() const override {
return settings_ & ZCR_EXTENDED_DRAG_V1_OPTIONS_ALLOW_DROP_NO_TARGET;
}
bool ShouldLockCursor() const override {
return settings_ & ZCR_EXTENDED_DRAG_V1_OPTIONS_LOCK_CURSOR;
}
void OnSwallowed(std::string mime_type) override {
zcr_extended_drag_source_v1_send_swallow(resource_, mime_type.c_str());
wl_client_flush(wl_resource_get_client(resource_));
}
void OnUnswallowed(std::string mime_type,
const gfx::Vector2d& offset) override {
zcr_extended_drag_source_v1_send_unswallow(resource_, mime_type.c_str(),
offset.x(), offset.y());
wl_client_flush(wl_resource_get_client(resource_));
}
void OnDataSourceDestroying() override { delete this; }
private:
wl_resource* const resource_;
const uint32_t settings_;
};
void extended_drag_source_destroy(wl_client* client, wl_resource* resource) {
wl_resource_destroy(resource);
}
void extended_drag_source_drag(wl_client* client,
wl_resource* resource,
wl_resource* surface_resource,
int32_t x_offset,
int32_t y_offset) {
Surface* surface =
surface_resource ? GetUserDataAs<Surface>(surface_resource) : nullptr;
gfx::Vector2d offset{x_offset, y_offset};
GetUserDataAs<ExtendedDragSource>(resource)->Drag(surface, offset);
}
const struct zcr_extended_drag_source_v1_interface
extended_drag_source_implementation = {extended_drag_source_destroy,
extended_drag_source_drag};
////////////////////////////////////////////////////////////////////////////////
// zcr_extended_drag_offer interface:
class ZcrExtendedOfferDelegate : public ExtendedDragOffer::Delegate {
public:
explicit ZcrExtendedOfferDelegate(wl_resource* resource)
: resource_(resource) {
DCHECK(resource_);
}
ZcrExtendedOfferDelegate(const ZcrExtendedOfferDelegate&) = delete;
ZcrExtendedOfferDelegate& operator=(const ZcrExtendedOfferDelegate&) = delete;
~ZcrExtendedOfferDelegate() override = default;
// ExtendedDragOffer::Delegate:
void OnDataOfferDestroying() override { delete this; }
private:
wl_resource* const resource_;
};
void extended_drag_offer_destroy(wl_client* client, wl_resource* resource) {
wl_resource_destroy(resource);
}
void extended_drag_offer_swallow(wl_client* client,
wl_resource* resource,
uint32_t serial,
const char* mime_type) {
GetUserDataAs<ExtendedDragOffer>(resource)->Swallow(serial, mime_type);
}
void extended_drag_offer_unswallow(wl_client* client,
wl_resource* resource,
uint32_t serial,
const char* mime_type,
int32_t x_offset,
int32_t y_offset) {
gfx::Vector2d offset{x_offset, y_offset};
GetUserDataAs<ExtendedDragOffer>(resource)->Unswallow(serial, mime_type,
offset);
}
const struct zcr_extended_drag_offer_v1_interface
extended_drag_offer_implementation = {extended_drag_offer_destroy,
extended_drag_offer_swallow,
extended_drag_offer_unswallow};
////////////////////////////////////////////////////////////////////////////////
// zcr_extended_drag interface:
void extended_drag_get_extended_drag_source(wl_client* client,
wl_resource* resource,
uint32_t id,
wl_resource* data_source_resource,
uint32_t settings) {
Display* display = GetUserDataAs<Display>(resource);
DataSource* source = GetUserDataAs<DataSource>(data_source_resource);
wl_resource* extended_drag_source_resource =
wl_resource_create(client, &zcr_extended_drag_source_v1_interface,
wl_resource_get_version(resource), id);
SetImplementation(extended_drag_source_resource,
&extended_drag_source_implementation,
std::make_unique<ExtendedDragSource>(
source, display->seat(),
new ZcrExtendedDragSourceDelegate(
extended_drag_source_resource, settings)));
}
void extended_drag_get_extended_drag_offer(wl_client* client,
wl_resource* resource,
uint32_t id,
wl_resource* data_offer_resource) {
DataOffer* offer = GetUserDataAs<DataOffer>(data_offer_resource);
wl_resource* extended_drag_offer_resource =
wl_resource_create(client, &zcr_extended_drag_offer_v1_interface,
wl_resource_get_version(resource), id);
SetImplementation(
extended_drag_offer_resource, &extended_drag_offer_implementation,
std::make_unique<ExtendedDragOffer>(
offer, new ZcrExtendedOfferDelegate(extended_drag_offer_resource)));
}
const struct zcr_extended_drag_v1_interface extended_drag_implementation = {
extended_drag_get_extended_drag_source,
extended_drag_get_extended_drag_offer};
} // namespace
void bind_extended_drag(wl_client* client,
void* data,
uint32_t version,
uint32_t id) {
wl_resource* resource =
wl_resource_create(client, &zcr_extended_drag_v1_interface, version, id);
wl_resource_set_implementation(resource, &extended_drag_implementation, data,
nullptr);
}
} // namespace wayland
} // namespace exo
// Copyright 2020 The Chromium Authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.
#ifndef COMPONENTS_EXO_WAYLAND_ZCR_EXTENDED_DRAG_H_
#define COMPONENTS_EXO_WAYLAND_ZCR_EXTENDED_DRAG_H_
#include <cstdint>
struct wl_client;
namespace exo {
namespace wayland {
void bind_extended_drag(wl_client* client,
void* data,
uint32_t version,
uint32_t id);
} // namespace wayland
} // namespace exo
#endif // COMPONENTS_EXO_WAYLAND_ZCR_EXTENDED_DRAG_H_
......@@ -123,3 +123,7 @@ wayland_protocol("color_space_protocol") {
wayland_protocol("xdg_foreign") {
sources = [ "src/unstable/xdg-foreign/xdg-foreign-unstable-v1.xml" ]
}
wayland_protocol("extended_drag") {
sources = [ "unstable/extended-drag/extended-drag-unstable-v1.xml" ]
}
Extended Drag protocol
Maintainers:
Nick Yamane <nickdiego@igalia.com>
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment