README.md 14.1 KB
Newer Older
1 2
![Omnibus Icon](lib/omnibus/assets/README-logo.png) Omnibus
===========================================================
Seth Vargo's avatar
Seth Vargo committed
3
[![Gem Version](http://img.shields.io/gem/v/omnibus.svg)][gem]
Scott Hain's avatar
Scott Hain committed
4
[![Build Status](http://img.shields.io/travis/chef/omnibus.svg)][travis]
Seth Vargo's avatar
Seth Vargo committed
5 6

[gem]: https://rubygems.org/gems/omnibus
Scott Hain's avatar
Scott Hain committed
7
[travis]: http://travis-ci.org/chef/omnibus
8

9
Easily create full-stack installers for your project across a variety of platforms.
Christopher Maier's avatar
Christopher Maier committed
10

11 12 13
Seth Chisamore and Christopher Maier of CHEF gave an introductory talk on Omnibus at ChefConf 2013, entitled **Eat the Whole Bowl: Building a Full-Stack Installer with Omnibus**:
  - [Video](http://www.youtube.com/watch?v=q8iJAntXCNY)
  - [Slides](https://speakerdeck.com/schisamo/eat-the-whole-bowl-building-a-full-stack-installer-with-omnibus)
14

15 16
This project is managed by the CHEF Release Engineering team. For more information on the Release Engineering team's contribution, triage, and release process, please consult the [CHEF Release Engineering OSS Management Guide](https://docs.google.com/a/opscode.com/document/d/1oJB0vZb_3bl7_ZU2YMDBkMFdL-EWplW1BJv_FXTUOzg/edit).

Stephen Delano's avatar
Stephen Delano committed
17

18 19 20 21 22
Omnibus fork notes
------------------

This fork contains omnibus-gitlab specific fixes. Changes are located at `gitlab_omnibus` branch.

23 24 25
Prerequisites
-------------
Omnibus is designed to run with a minimal set of prerequisites. You will need the following:
Stephen Delano's avatar
Stephen Delano committed
26

Kartik Null Cating-Subramanian's avatar
Kartik Null Cating-Subramanian committed
27
- Ruby 2.0.0+
28
- Bundler
Christopher Maier's avatar
Christopher Maier committed
29

Stephen Delano's avatar
Stephen Delano committed
30

31 32 33
Get Started
-----------
Omnibus provides both a DSL for defining Omnibus projects for your software, as well as a command-line tool for generating installer artifacts from that definition.
34

Christopher Maier's avatar
Christopher Maier committed
35
To get started, install Omnibus locally on your workstation.
36

37
```bash
Christopher Maier's avatar
Christopher Maier committed
38 39
$ gem install omnibus
```
40

41
You can now create an Omnibus project in your current directory by using the project generator feature.
42

43
```bash
44
$ omnibus new $MY_PROJECT_NAME
45 46
```

47
This will generate a complete project skeleton in the directory `omnibus-$MY_PROJECT_NAME`
Christopher Maier's avatar
Christopher Maier committed
48

49
```bash
50
$ cd omnibus-$MY_PROJECT_NAME
Christopher Maier's avatar
Christopher Maier committed
51
$ bundle install --binstubs
52
$ bin/omnibus build $MY_PROJECT_NAME
53 54
```

55 56 57
More details can be found in the generated project's README file.

Omnibus determines the platform for which to build an installer based on **the platform it is currently running on**. That is, you can only generate a `.deb` file on a Debian-based system. To alleviate this caveat, the generated project includes a [Test Kitchen](http://kitchen.ci) setup suitable for generating a series of Omnibus projects.
Stephen Delano's avatar
Stephen Delano committed
58

Seth Vargo's avatar
Seth Vargo committed
59

60 61
More documentation
------------------
62
- [Building on Debian](docs/Building on Debian.md)
63 64
- [Building on OSX](docs/Building on OSX.md)
- [Building on RHEL](docs/Building on RHEL.md)
65
- [Building on Windows](docs/Building on Windows.md)
66
- [Build Cache](docs/Build Cache.md)
67

Stephen Delano's avatar
Stephen Delano committed
68

69
Configuration DSL
70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92
-----------------
Though the template project will build, it will not do anything exciting. For that, you need to use the Omnibus DSL to define the specifics of your application.

### Config
If present, Omnibus will use a top-level configuration file named `omnibus.rb` at the root of your repository. This file is loaded at runtime and includes a number of configuration tunables. Here is an example:

```ruby
# Build locally (instead of /var)
# -------------------------------
base_dir './local'

# Disable git caching
# ------------------------------
use_git_caching false

# Enable S3 asset caching
# ------------------------------
use_s3_caching true
s3_access_key  ENV['S3_ACCESS_KEY']
s3_secret_key  ENV['S3_SECRET_KEY']
s3_bucket      ENV['S3_BUCKET']
```

Seth Vargo's avatar
Seth Vargo committed
93
For more information, please see the [`Config` documentation](http://rubydoc.info/github/opscode/omnibus/Omnibus/Config).
94

Michael Mior's avatar
Michael Mior committed
95
You can tell Omnibus to load a different configuration file by passing the `--config` option to any command:
96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112

```shell
$ bin/omnibus --config /path/to/config.rb
```

Finally, you can override a specific configuration option at the command line using the `--override` flag. This takes ultimate precedence over any configuration file values:

```shell
$ bin/omnibus --override use_git_caching:false
```

### Projects
A Project DSL file defines your actual application; this is the thing you are creating a full-stack installer for in the first place. It provides a means to define the dependencies of the project (again, as specified in Software DSL definition files), as well as ways to set installer package metadata.

All project definitions must be in the `config/projects` directory of your Omnibus repository.

```ruby
Seth Vargo's avatar
Seth Vargo committed
113 114 115
name            "chef-full"
maintainer      "YOUR NAME"
homepage        "http://yoursite.com"
116

Seth Vargo's avatar
Seth Vargo committed
117 118
install_dir     "/opt/chef"
build_version   "0.10.8"
119 120
build_iteration 4

Seth Vargo's avatar
Seth Vargo committed
121
dependency "chef"
122 123 124 125 126 127 128 129 130 131 132
```

Some DSL methods available include:

| DSL Method        | Description                                 |
| :---------------: | --------------------------------------------|
| `name`            | The name of the project                     |
| `install_dir`     | The desired install location of the package |
| `build_version`   | The package version                         |
| `build_iteration` | The package iteration number                |
| `dependency`      | An Omnibus software-defined component to include in this package |
133 134
| `package`         | Invoke a packager-specific DSL              |
| `compress`        | Invoke a compressor-specific DSL            |
135

136 137 138 139 140
By default a timestamp is appended to the build_version.  You can turn
this behavior off by setting `append_timestamp` to `false` in your
configuration file or using `--override append_timestamp:false` at the
command line.

Seth Vargo's avatar
Seth Vargo committed
141
For more information, please see the [`Project` documentation](http://rubydoc.info/github/opscode/omnibus/Omnibus/Project).
142

143
### Software
144
Omnibus "software" files define individual software components that go into making your overall package. They are the building blocks of your application. The Software DSL provides a way to define where to retrieve the software sources, how to build them, and what dependencies they have. These dependencies are also defined in their own Software DSL files, thus forming the basis for a dependency-aware build ordering.
Stephen Delano's avatar
Stephen Delano committed
145

146 147
All Software definitions should go in the `config/software` directory of your Omnibus project repository.

148
Here is an example:
Christopher Maier's avatar
Christopher Maier committed
149

150
```ruby
Seth Vargo's avatar
Seth Vargo committed
151 152 153 154
name "ruby"
default_version "1.9.2-p290"
source url: "http://ftp.ruby-lang.org/pub/ruby/1.9/ruby-#{version}.tar.gz",
       md5: "604da71839a6ae02b5b5b5e1b792d5eb"
155

Seth Vargo's avatar
Seth Vargo committed
156 157 158
dependency "zlib"
dependency "ncurses"
dependency "openssl"
Stephen Delano's avatar
Stephen Delano committed
159

160
relative_path "ruby-#{version}"
Stephen Delano's avatar
Stephen Delano committed
161

162
build do
Seth Vargo's avatar
Seth Vargo committed
163 164 165
  command "./configure"
  command "make"
  command "make install"
166 167
end
```
Stephen Delano's avatar
Stephen Delano committed
168

Christopher Maier's avatar
Christopher Maier committed
169 170
Some of the DSL methods available include:

171 172 173 174 175 176 177 178
| DSL Method        | Description                                |
| :---------------: | -------------------------------------------|
| `name`            | The name of the software component (this should come first) |
| `default_version` | The version of the software component      |
| `source`          | Directions to the location of the source   |
| `dependency`      | An Omnibus software-defined component that this software depends on |
| `relative_path`   | The relative path of the extracted tarball |
| `build`           | The build instructions                     |
179 180 181 182 183 184 185 186 187 188

For more DSL methods, please consult the [`Software` documentation](http://rubydoc.info/github/opscode/omnibus/Omnibus/Software).

Additionally, there are a number of DSL methods avaiable inside the `build` block:

| DSL Method          | Description                                |
| :-----------------: | -------------------------------------------|
| `command`           | Execute a single shell command             |
| `make`              | Run make (with or without args), using gmake when appropriate |
| `patch`             | Apply a patch from disk                    |
189
| `workers`           | The maximum number of builders             |
190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205
| `windows_safe_path` | Format the path to be safe for shelling out on Windows |
| `ruby`              | Execute the code as the embedded Ruby      |
| `gem`               | Execute the code as the embedded Rubygems  |
| `bundle`            | Execute the code as the embedded Bundler   |
| `rake`              | Execute the code as the embedded Rake gem  |
| `block`             | Execute Ruby block at build time           |
| `erb`               | Render the given ERB template              |
| `mkdir`             | Create the given directory                 |
| `touch`             | Create the given empty file                |
| `delete`            | Remove the given file or directory         |
| `copy`              | Copy a to b                                |
| `move`              | Move a to b                                |
| `link`              | Link a to b                                |
| `sync`              | Copy all files from a to b, removing any union files |

For more DSL methods, please consult the [`Builder` documentation](http://rubydoc.info/github/opscode/omnibus/Omnibus/Builder).
Stephen Delano's avatar
Stephen Delano committed
206

207
You can support building multiple verisons of the same software in the same software definition file using the `version` method and giving a block:
Stephen Delano's avatar
Stephen Delano committed
208

209
```ruby
Seth Vargo's avatar
Seth Vargo committed
210 211
name "ruby"
default_version "1.9.2-p290"
Stephen Delano's avatar
Stephen Delano committed
212

Seth Vargo's avatar
Seth Vargo committed
213 214 215
version "1.9.2-p290" do
  source url: "http://ftp.ruby-lang.org/pub/ruby/1.9/ruby-#{version}.tar.gz",
         md5: "604da71839a6ae02b5b5b5e1b792d5eb"
216
end
217

Seth Vargo's avatar
Seth Vargo committed
218 219 220
version "2.1.1" do
  source url: "http://ftp.ruby-lang.org/pub/ruby/2.1/ruby-#{version}.tar.gz",
         md5: "e57fdbb8ed56e70c43f39c79da1654b2"
221 222
end
```
223

224
Since the software definitions are simply ruby code, you can conditionally execute anything by wrapping it with pure Ruby that tests for the version number.
225

226
#### Sharing software definitions
Scott Hain's avatar
Scott Hain committed
227
The easiest way to share organization-wide software is via bundler and Rubygems. For an example software repository, look at Chef's [omnibus-software](https://github.com/chef/omnibus-software). For more information, please see the [Rubygems documentation](http://guides.rubygems.org/publishing/).
228

229
It is recommended you use bundler to pull down these gems (as bundler also permits pulling software directly from GitHub):
Christopher Maier's avatar
Christopher Maier committed
230

231
```ruby
232 233 234
gem 'my-company-omnibus-software'
gem 'omnibus-software', github: 'my-company/omnibus-software'
```
235

236
Then add the name of the software to the list of `software_gems` in your Omnibus config:
237

238 239
```ruby
software_gems %w(my-company-omnibus-software omnibus-software)
240 241
```

242
You may also specify local paths on disk (but be warned this may make sharing the project among teams difficult):
Christopher Maier's avatar
Christopher Maier committed
243

244 245 246
```ruby
local_software_dirs %w(/path/to/software /other/path/to/software)
```
247

248
For all of these paths, **order matters**, so it is possible to depend on local software version while still retaining a remote software repo. Given the above example, Omnibus will search for a software definition named `foo` in this order:
249 250


251 252 253 254 255 256 257
```text
$PWD/config/software/foo.rb
/path/to/software/config/software/foo.rb
/other/path/to/software/config/software/foo.rb
/Users/sethvargo/.gems/.../my-comany-omnibus-software/config/software/foo.rb
/Users/sethvargo/.gems/.../omnibus-software/config/software/foo.rb
```
258

259
The first instance of `foo.rb` that is encountered will be used. Please note that **local** (vendored) softare definitions take precedence!
260

261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276
Version Manifest
----------------

Git-based software definitions may specify branches as their
default_version. In this case, the exact git revision to use will be
determined at build-time unless a project override (see below) or
external version manifest is used. To generate a version manifest use
the `omnibus manifest` command:

```
omnibus manifest PROJECT -l warn
```

This will output a JSON-formatted manifest containing the resolved
version of every software definition.

277 278 279 280 281 282 283 284 285 286 287 288 289 290
Whitelisting Libraries
----------------------

Sometimes a platform has libraries that need to be whitelisted so the healthcheck
can pass. The whitelist found in the [healthcheck](https://github.com/chef/omnibus/blob/master/lib/omnibus/health_check.rb)
code comprises the minimal required for successful builds on supported platforms.

To add your own whitelisted library, simply add the a regex to your software
definition in your omnibus project as follows:
```
whitelist_file /libpcrecpp\.so\..+/
```
It is typically a good idea to add a conditional to whitelist based on the specific
platform that requires it.
291

Scott Hain's avatar
Scott Hain committed
292 293 294
*Warning: You should only add libraries to the whitelist that are guaranteed to
be on the system you install to; if a library comes from a non-default package
you should instead build it into the package.*
295

296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312
Changelog
---------
STATUS: *EXPERIMENTAL*

`omnibus changelog generate` will generate a changelog for an omnibus
project.  This command currently assumes:

- version-manifest.json is checked into the project root
- the project is a git repository
- each version is tagged with a SemVer compliant annotated tag
- Any git-based sources are checked out at ../COMPONENT_NAME
- Any commit message line prepended with ChangeLog-Entry: should be
  added to the changelog.

These assumptions *will* change as we determine what works best for a
number of our projects.

313

314 315
Caveats
-------
316 317
### Overrides
The project definitions can override specific software dependencies by passing in `override` to use the correct version:
318 319

```ruby
Seth Vargo's avatar
Seth Vargo committed
320
name "chef-full"
321
# <snip>
322

323
# This will override the default version of "chef"
Seth Vargo's avatar
Seth Vargo committed
324
override :chef, version: "2.1.1"
325

Seth Vargo's avatar
Seth Vargo committed
326
dependency "chef"
327 328
```

329 330 331 332 333 334
**The overridden version must be defined in the associated software!**

### Debugging
By default, Omnibus will log at the `warn` level. You can override this by passing the `--log-level` flag to your Omnibus call:

```shell
Seth Vargo's avatar
Seth Vargo committed
335
$ bin/omnibus build <project> --log-level info # or "debug"
336
```
337

338
### Git caching
339
by default, Omnibus caches compiled software definitions, so n+1 Omnibus project builds are much faster. This functionality can be disabled by adding the following to your `omnibus.rb`:
340

341 342 343
```ruby
use_git_caching false
```
344 345


346 347 348 349
License
-------
```text
Copyright 2012-2014 Chef Software, Inc.
350 351 352 353 354 355 356 357 358 359 360

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
361
limitations under the License.
362
```