Upstream helper script to build Chromium on iOS and update instructions.

Upstream some scripts used to simplify the setup of a new Chromium on iOS
checkout and update the instruction to remove obsolete informations, and
add recommendation to use the script.

BUG=None

Review-Url: https://codereview.chromium.org/2343853002
Cr-Commit-Position: refs/heads/master@{#419422}

docs/ios_build_instructions.md

@@ -7,97 +7,72 @@ the web layer), and various unit tests.
 
 ## Prerequisites
 
-*   A Mac with a version of OS X capable of running the latest version
-    of Xcode.
-*   The latest version of [Xcode](https://developer.apple.com/xcode/),
-    including the current iOS SDK.
+*   A Mac running 10.11+.
+*   [Xcode] 8.0+.
+*   [depot\_tools].
 *   The current version of the JDK (required for the closure compiler).
-*   [depot\_tools](http://dev.chromium.org/developers/how-tos/install-depot-tools).
 
-## Setting Up
+## Getting the source
 
-### With GYP
-
-In the directory where you are going to check out the code, create a
-`chromium.gyp_env` to set the build to use iOS targets (and to use
-hybrid builds; see [Building](#Building) below):
+To checkout the source, use `fetch ios` command from [depot\_tools] in a new
+empty directory.
 
 ```shell
-cat > chromium.gyp_env <<EOF
-{
-  "GYP_DEFINES": "OS=ios",
-  "GYP_GENERATORS": "ninja,xcode-ninja",
-}
-EOF
+# You can use a different location for your checkout of Chromium on iOS
+# by updating this variable. All shell snippets will refer to it.
+CHROMIUM_IOS="$HOME/chromium_ios"
+mkdir "$CHROMIUM_IOS"
+cd "$CHROMIUM_IOS"
+fetch ios
 ```
 
-If you aren't set up to sign iOS build products via a developer account,
-you should instead use:
+## Setting up
 
-```shell
-cat > chromium.gyp_env <<EOF
-{
-  "GYP_DEFINES": "OS=ios chromium_ios_signing=0",
-  "GYP_GENERATORS": "ninja,xcode-ninja",
-}
-EOF
-```
+Chromium on iOS is built using the [Ninja](ninja_build.md) tool and
+the [Clang](clang.md) compiler. See both of those pages for further details on
+how to tune the build.
 
-### With GN
+Before you build, you may want to [install API keys](api-keys) so that
+Chrome-integrated Google services work. This step is optional if you aren't
+testing those features.
 
-Use `gn args out/Debug-iphonesimulator` (or replace
-`out/Debug-iphonesimulator` with your chosen `out/` directory) to open up an
-editor to set the following gn variables and regenerate:
+### Quick setup
 
-```
-# Set to true if you have a valid code signing key.
-ios_enable_code_signing = false
-target_os = "ios"
-# Set to "x86", "x64", "arm", "armv7", "arm64". "x86" and "x64" will create a
-# build to run on the iOS simulator (and set use_ios_simulator = true), all
-# others are for an iOS device.
-target_cpu = "x64"
-# Release vs debug build.
-is_debug = true
-```
-
-### API Keys
-
-Before you build, you may want to
-[install API keys](https://sites.google.com/a/chromium.org/dev/developers/how-tos/api-keys)
-so that Chrome-integrated Google services work. This step is optional if you
-aren't testing those features.
-
-## Getting the Code
-
-Next, [check out the
-code](https://www.chromium.org/developers/how-tos/get-the-code), with:
+To setup the repository for building Chromium on iOS code, it is recommended
+to use the `src/ios/build/tools/setup-gn.py` script that creates a Xcode
+workspace configured to build the different targets for device and simulator.
 
 ```shell
-fetch ios
+cd "$CHROMIUM_IOS/src"
+ios/build/tools/setup-gn.py
+open out/build/all.xcworkspace
 ```
 
-## Building
+You can customize the build by editing the file `$HOME/.setup-gn` (create it
+if it does not exists).  Look at `src/ios/build/tools/setup-gn.config` for
+available configuration options.
 
-Build the target you are interested in. The instructions above select
-the ninja/Xcode hybrid mode, which uses ninja to do the actual build,
-but provides a wrapper Xcode project that can be used to build targets
-and navigate the source. (The Xcode project just shells out to ninja to
-do the builds, so you can't actually inspect/change target-level
-settings from within Xcode; this mode avoids generating a large tree of
-Xcode projects, which leads to performance issues in Xcode). To build
-with ninja (simulator and device, respectively):
+From this point, you can either build from Xcode or from the command-line
+using `ninja`. The script `setup-gn.py` creates sub-directories named
+`out/${configuration}-${platform}`, so for a `Debug` build for simulator
+use:
 
 ```shell
-ninja -C out/Debug-iphonesimulator All
-ninja -C out/Debug-iphoneos All
+ninja -C out/Debug-iphonesimulator gn_all
 ```
 
-To build with Xcode, open `build/all.ninja.xcworkspace`, and choose the
-target you want to build.
+Note: you need to run `setup-gn.py` script every time one of the `BUILD.gn`
+file is updated (either by you or after rebasing). If you forget to run it,
+the list of targets and files in the Xcode solution may be stale.
 
-You should always be able to build All, since targets are added there for iOS
-only when they compile.
+### Advanced setup
+
+You can run `gn` manually to configure the build yourself. In that case,
+refer to [mac build instructions] for help on how to do that.
+
+To build for iOS, you have to set `target_os` to `"ios"`. Please also note
+that `is_component_build` is not supported when building for iOS and must
+be set to `false`.
 
 ## Running
 
@@ -114,37 +89,16 @@ example, to run a debug build of ios\_web\_shell:
 out/Debug-iphonesimulator/iossim out/Debug-iphonesimulator/ios_web_shell.app
 ```
 
-## Converting an existing Mac checkout into an iOS checkout
-
-If you want to convert your Mac checkout into an iOS checkout, follow the steps
-below:
-
-1.  Add `target_os = [ "ios" ]` to the bottom of your `chromium/.gclient`
-file.
-
-2.  For gyp, make sure you have the following in your
-`chromium/chromium.gyp_env` file (removing the `chromium_ios_signing=0` if you
-want to make developer-signed builds):
-
-    ```json
-    {
-      "GYP_DEFINES" : "OS=ios chromium_ios_signing=0",
-      "GYP_GENERATORS" : "ninja,xcode-ninja",
-    }
-    ```
-
-    For gn, add the arguments specified [above](#With-GN) to your gn setup.
-
-3.  Make sure to sync again to fetch the iOS specific dependencies and
-regenerate build rules using:
-
-    ```shell
-    gclient sync
-    ```
-
 ## Troubleshooting
 
 If your build fails, check the iOS columns of [the Mac
 waterfall](http://build.chromium.org/p/chromium.mac/console) (the last two) to
 see if the bots are green. In general they should be, since failures on those
 bots will close the tree.
+
+[Xcode]: https://developer.apple.com/xcode
+[depot\_tools]: https://dev.chromium.org/developers/how-tos/depottools
+[Ninja]: ninja.md
+[Clang]: clang.md
+[api-keys]: https://sites.google.com/a/chromium.org/dev/developers/how-tos/api-keys
+[mac build instructions]: mac_build_instructions.md

ios/build/tools/convert_gn_xcodeproj.py

+#!/usr/bin/python
+# Copyright 2016 The Chromium Authors. All rights reserved.
+# Use of this source code is governed by a BSD-style license that can be
+# found in the LICENSE file.
+
+"""Convert GN Xcode projects to platform and configuration independent targets.
+
+GN generates Xcode projects that build one configuration only. However, typical
+iOS development involves using the Xcode IDE to toggle the platform and
+configuration. This script replaces the 'gn' configuration with 'Debug',
+'Release' and 'Profile', and changes the ninja invokation to honor these
+configurations.
+"""
+
+import argparse
+import collections
+import copy
+import filecmp
+import json
+import hashlib
+import os
+import plistlib
+import random
+import shutil
+import subprocess
+import sys
+import tempfile
+
+
+XCTEST_PRODUCT_TYPE = 'com.apple.product-type.bundle.unit-test'
+
+
+class XcodeProject(object):
+
+  def __init__(self, objects, counter = 0):
+    self.objects = objects
+    self.counter = 0
+
+  def AddObject(self, parent_name, obj):
+    while True:
+      self.counter += 1
+      str_id = "%s %s %d" % (parent_name, obj['isa'], self.counter)
+      new_id = hashlib.sha1(str_id).hexdigest()[:24].upper()
+
+      # Make sure ID is unique. It's possible there could be an id conflict
+      # since this is run after GN runs.
+      if new_id not in self.objects:
+        self.objects[new_id] = obj
+        return new_id
+
+
+def CopyFileIfChanged(source_path, target_path):
+  """Copy |source_path| to |target_path| is different."""
+  target_dir = os.path.dirname(target_path)
+  if not os.path.isdir(target_dir):
+    os.makedirs(target_dir)
+  if not os.path.exists(target_path) or \
+      not filecmp.cmp(source_path, target_path):
+    shutil.copyfile(source_path, target_path)
+
+
+def LoadXcodeProjectAsJSON(path):
+  """Return Xcode project at |path| as a JSON string."""
+  return subprocess.check_output([
+      'plutil', '-convert', 'json', '-o', '-', path])
+
+
+def WriteXcodeProject(output_path, json_string):
+  """Save Xcode project to |output_path| as XML."""
+  with tempfile.NamedTemporaryFile() as temp_file:
+    temp_file.write(json_string)
+    temp_file.flush()
+    subprocess.check_call(['plutil', '-convert', 'xml1', temp_file.name])
+    CopyFileIfChanged(temp_file.name, output_path)
+
+
+def UpdateProductsProject(file_input, file_output, configurations):
+  """Update Xcode project to support multiple configurations.
+
+  Args:
+    file_input: path to the input Xcode project
+    file_output: path to the output file
+    configurations: list of string corresponding to the configurations that
+      need to be supported by the tweaked Xcode projects, must contains at
+      least one value.
+  """
+  json_data = json.loads(LoadXcodeProjectAsJSON(file_input))
+  project = XcodeProject(json_data['objects'])
+
+  objects_to_remove = []
+  for value in project.objects.values():
+    isa = value['isa']
+
+    # TODO(crbug.com/619072): gn does not write the min deployment target in the
+    # generated Xcode project, so add it while doing the conversion, only if it
+    # is not present. Remove this code and comment once the bug is fixed and gn
+    # has rolled past it.
+    if isa == 'XCBuildConfiguration':
+      build_settings = value['buildSettings']
+      if 'IPHONEOS_DEPLOYMENT_TARGET' not in build_settings:
+        build_settings['IPHONEOS_DEPLOYMENT_TARGET'] = '9.0'
+
+    # Remove path name key and change path to basename.
+    if isa == 'PBXFileReference':
+      if 'name' in value:
+        del value['name']
+      value['path'] = os.path.basename(value['path'])
+
+    # Teach build shell script to look for the configuration and platform.
+    if isa == 'PBXShellScriptBuildPhase':
+      value['shellScript'] = value['shellScript'].replace(
+          'ninja -C .',
+          'ninja -C "../${CONFIGURATION}${EFFECTIVE_PLATFORM_NAME}"')
+
+    # Configure BUNDLE_LOADER and TEST_HOST for xctest target (assuming that
+    # the host is named "${target}_host") unless gn has already configured
+    # them.
+    if isa == 'PBXNativeTarget' and value['productType'] == XCTEST_PRODUCT_TYPE:
+      configuration_list = project.objects[value['buildConfigurationList']]
+      for config_name in configuration_list['buildConfigurations']:
+        config = project.objects[config_name]
+        if not config['buildSettings'].get('BUNDLE_LOADER'):
+          config['buildSettings']['BUNDLE_LOADER'] = '$(TEST_HOST)'
+          config['buildSettings']['TEST_HOST'] = \
+              '${BUILT_PRODUCTS_DIR}/%(name)s_host.app/%(name)s' % value
+
+    # Add new configuration, using the first one as default.
+    if isa == 'XCConfigurationList':
+      value['defaultConfigurationName'] = configurations[0]
+      objects_to_remove.extend(value['buildConfigurations'])
+
+      build_config_template = project.objects[value['buildConfigurations'][0]]
+      build_config_template['buildSettings']['CONFIGURATION_BUILD_DIR'] = \
+          '../$(CONFIGURATION)$(EFFECTIVE_PLATFORM_NAME)'
+
+      value['buildConfigurations'] = []
+      for configuration in configurations:
+        new_build_config = copy.copy(build_config_template)
+        new_build_config['name'] = configuration
+        value['buildConfigurations'].append(
+            project.AddObject('products', new_build_config))
+
+  for object_id in objects_to_remove:
+    del project.objects[object_id]
+
+  objects = collections.OrderedDict(sorted(project.objects.iteritems()))
+  WriteXcodeProject(file_output, json.dumps(json_data))
+
+
+def ConvertGnXcodeProject(input_dir, output_dir, configurations):
+  '''Tweak the Xcode project generated by gn to support multiple configurations.
+
+  The Xcode projects generated by "gn gen --ide" only supports a single
+  platform and configuration (as the platform and configuration are set
+  per output directory). This method takes as input such projects and
+  add support for multiple configurations and platforms (to allow devs
+  to select them in Xcode).
+
+  Args:
+    input_dir: directory containing the XCode projects created by "gn gen --ide"
+    output_dir: directory where the tweaked Xcode projects will be saved
+    configurations: list of string corresponding to the configurations that
+      need to be supported by the tweaked Xcode projects, must contains at
+      least one value.
+  '''
+  # Update products project.
+  products = os.path.join('products.xcodeproj', 'project.pbxproj')
+  product_input = os.path.join(input_dir, products)
+  product_output = os.path.join(output_dir, products)
+  UpdateProductsProject(product_input, product_output, configurations)
+
+  # Copy sources project and all workspace.
+  sources = os.path.join('sources.xcodeproj', 'project.pbxproj')
+  CopyFileIfChanged(os.path.join(input_dir, sources),
+                    os.path.join(output_dir, sources))
+  xcwspace = os.path.join('all.xcworkspace', 'contents.xcworkspacedata')
+  CopyFileIfChanged(os.path.join(input_dir, xcwspace),
+                    os.path.join(output_dir, xcwspace))
+
+
+def Main(args):
+  parser = argparse.ArgumentParser(
+      description='Convert GN Xcode projects for iOS.')
+  parser.add_argument(
+      'input',
+      help='directory containing [product|sources|all] Xcode projects.')
+  parser.add_argument(
+      'output',
+      help='directory where to generate the iOS configuration.')
+  parser.add_argument(
+      '--add-config', dest='configurations', default=[], action='append',
+      help='configuration to add to the Xcode project')
+  args = parser.parse_args(args)
+
+  if not os.path.isdir(args.input):
+    sys.stderr.write('Input directory does not exists.\n')
+    return 1
+
+  required = set(['products.xcodeproj', 'sources.xcodeproj', 'all.xcworkspace'])
+  if not required.issubset(os.listdir(args.input)):
+    sys.stderr.write(
+        'Input directory does not contain all necessary Xcode projects.\n')
+    return 1
+
+  if not args.configurations:
+    sys.stderr.write('At least one configuration required, see --add-config.\n')
+    return 1
+
+  ConvertGnXcodeProject(args.input, args.output, args.configurations)
+
+if __name__ == '__main__':
+  sys.exit(Main(sys.argv[1:]))
+
+

ios/build/tools/setup-gn.config

+[goma]
+# Controls whether goma is enabled or not. If you generally use goma but
+# want to disable goma for a single build, consider using the environment
+# variable GOMA_DISABLED.
+enabled = False
+install = "$GOMA_DIR"
+
+[xcode]
+# Controls settings for the generated Xcode project. If jobs is non-zero
+# it will be passed to the ninja invocation in Xcode project.
+jobs = 0
+
+[build]
+# Controls the build output. The only supported values are "64-bit", "32-bit"
+# and "multi" (for a fat binary supporting both "32-bit" and "64-bit" cpus).
+arch = "64-bit"
+
+[gn_args]
+# Values in that section will be copied verbatim in the generated args.gn file.
+target_os = "ios"
+
+[filters]
+# List of target files to pass to --filters argument of gn gen when generating
+# the Xcode project. By default, list all targets from ios/ and ios_internal/
+# and the targets corresponding to the unit tests run on the bots.
+filter_0 = "//base:base_unittests"
+filter_1 = "//components:components_unittests"
+filter_2 = "//crypto:crypto_unittests"
+filter_3 = "//google_apis:google_apis_unittests"
+filter_4 = "//ios/*"
+filter_5 = "//ios_internal/*"
+filter_6 = "//net:net_unittests"
+filter_7 = "//skia:skia_unittests"
+filter_8 = "//sql:sql_unittests"
+filter_9 = "//ui/base:ui_base_unittests"
+filter_a = "//ui/gfx:gfx_unittests"
+filter_b = "//url:url_unittests"