Commit dc6b89b2 authored by Marin Jankovski's avatar Marin Jankovski

Merge tag 'v4.1.0' of https://github.com/chef/omnibus into upgrade_omnibus_version

Version 4.1.0
parents 1b336f74 27d41edb
...@@ -83,6 +83,14 @@ Gemfile.lock ...@@ -83,6 +83,14 @@ Gemfile.lock
*.tmproject *.tmproject
tmtags tmtags
# avoid generic ctags files too
tags
gems.tags
gemtags
.tags
.gem.tags
.gemtags
.tags_sorted_by_file
### SublimeText ### ### SublimeText ###
# workspace files are user-specific # workspace files are user-specific
......
rvm: rvm:
- 1.9.3
- 2.0.0 - 2.0.0
- 2.1 - 2.1
bundler_args: "--jobs 7 --without docs local"
bundler_args: --jobs 7 --without docs local
branches: branches:
only: only:
- 2.0-stable - 2.0-stable
- 3.0-stable - 3.0-stable
- master - master
script: bundle exec rake travis:ci script: bundle exec rake travis:ci
notifications: notifications:
hipchat: slack:
on_change: true on_change: true
on_failure: true on_failure: true
on_success: false on_success: false
on_pull_requests: false on_pull_requests: false
rooms: rooms:
# Build Statuses secure: udLCoBl71xJBsW3P+YqZbA5xfaB8l4IIF6SKEXpjIUYNYyP6m77uoY9PZ/M/5EN1ojPXDxsZ6SiPEJifVpkX5Hqd0eu1MwWyRXvOwiDf/nkg2Y3mqwUQRb3LDYWZxPn+xp5mHePe3mvTICR1zH49yx7fdZs8Qvzhh3Lno/w/qZ4=
- secure: JdLOITSPHW5xk8eoFOmH1Js5PT6iFgswUG8fIqNq69ie2Qws2K58hPDR6HW2NBEsBb7dW1S5jT6V9RHhm0ykekvEJVa5AjebO3EsZa+Cu/VahFMg4DL+SGYgKlKrZQosd+EgVQQ3C9gj0dApGtKqf2Ej7RNESwCPY8SOduTC6d8=
# Release Engineering
- secure: PmOaqp2DFyy79VpCuvSYILblK6tP0eDa9fDG9X2j+wggoosBkmlGfnSxp3A2TXlUyK62mr6/B89dRkL2aDWTJ/gqaZ34Elx2rV5S2A5YJIx9stA2+4iXeSlOF2YMNKGwBVmF/kp/nNoG2FqUkUBrRurWAOXZnwxKIhQ7+kLVb/0=
Omnibus CHANGELOG Omnibus CHANGELOG
================= =================
v4.1.0 (Unreleased) v4.1.0 (September 1, 2015)
-------------------------
### New Features
- Allow semver prefixes that start with "v"
- Reset target_shasum in PathFetcher after every fetch, preventing erroneous GitCache misses for builds that produced local artifacts during the previous omnibus run.
- Allow users to specify options to the underlying FileSyncer in a Software definition
- Add CPPFLAGS in with_standard_compiler_flags
- Add a Fedora dist tag to package name and spec release
- Copy distribution.xml.erb also when creating project with --pkg-assets
- Manage version manifests with Omnibus. Omnibus now creates a version manifest in text and JSON.
- Retry failed downloads
- Support inclusion of email address in Wix template
- Allow solaris to use mapfiles
- A new `omnibus generate changelog` command generates an opinionated CHANGELOG entry based on metadata in commit messages.
- Add warnings on empty globs.
- Raspberry Pi 2 support
- Clone git repositories with --recursive flag
- Add ability to sign MSIs
- Support running Omnibus with 64-bit ruby on Windows
- Create an Artifactory build for published packages
- Add a "windows_arch" omnibus option to choose 32/64 bit builds.
- Set directory ownership/permissions to match the filesystem package.
- Allow Windows 10 builders
- Allow omnibus to build "bundled" Windows installers
- Set perms on control files per Ubuntu Software Center's lintian checks.
- Replace uber-s3 dependency with aws-sdk
### Bug Fixes ### Bug Fixes
- Config.append_timestamp is now properly handled by the build_version - Config.append_timestamp is now properly handled by the build_version
DSL. For some users, this may introduce a change in behavior. To DSL. For some users, this may introduce a change in behavior. To
revert to the old behavior set append_timestamp to false in revert to the old behavior set append_timestamp to false in
`omnibus.rb` or use --override append_timestamp:false at the command `omnibus.rb` or use --override append_timestamp:false at the command
line. line.
- Do not memoize current_revision to avoid returning incorrect data. Memoizing current_revision was previously causing version_for_cache to return the pre-fetch revision of the software, leading us to erroneously restore an older revision from the cache.
- Clean up "files listed twice" warnings. This should allow signing RPMs on EL 7
- dpkg uses arch name ppc64el (instead of ppc64le) for little endian
- Override equality operator for ManifestEntry. Prior to this, we simply tested basic object identity, which would result in many "updated" dependencies
- Correctly determine git repository when running in a subdirectory
- Forcibly remove a non-empty project dir before cloning because if multiple projects are using the same maching for building, they might have two different software definitions that share the same name but a different source.
- Accomidate hardlinks properly
- When building spec file, handle path names that contain spaces
- Add libkvm and libprocstat to freebsd whitelist
- Properly render ERB templates that do not have variables
- Change git fetcher to correctly fetch when repo name is the same but branch/version is different.
### Potentially Breaking Changes
- `Omnibus::SemanticVersion.new` now raises `Omnibus::InvalidVersion` instead of `Omnibus::SemanticVersion::InvalidVersion`
- Cache Builder#shasum before Builder#build to ensure consistent result.
- Update chef-sugar to 3.0 which adds ppc64le support.
- The MSI packager now adds the architecture to the msi name. The file names go from `package_name-build_version-build_iteration` to `package_name-build_version-build_iteration-arch`.
v4.0.0 (December 15, 2014) v4.0.0 (December 15, 2014)
-------------------------- --------------------------
......
# Maintainers
This file lists how the Omnibus project is maintained. When making changes
to the system, this file tells you who needs to review your patch - you need a
simple majority of maintainers for the relevant subsystems to provide a :+1: on
your pull request. Additionally, you need to not receive a veto from a
Lieutenant or the Project Lead.
Check out
[How Chef is Maintained](https://github.com/chef/chef-rfc/blob/master/rfc030-maintenance-policy.md#how-the-project-is-maintained)
for details on the process, how to become a maintainer, lieutenant, or the
project lead.
## Project Lead
* [Seth Chisamore](https://github.com/schisamo)
## Maintainers
* [Daniel DeLeo](https://github.com/danielsdeleo)
* [Jay Mundrawala](https://github.com/jdmundrawala)
* [Lamont Granquist](https://github.com/lamont-granquist)
* [Scott Hain](https://github.com/scotthain)
* [Seth Vargo](http://github.com/sethvargo)
* [Steven Danna](https://github.com/stevendanna)
* [Yvonne Lam](http://github.com/yzl)
...@@ -24,7 +24,7 @@ Prerequisites ...@@ -24,7 +24,7 @@ Prerequisites
------------- -------------
Omnibus is designed to run with a minimal set of prerequisites. You will need the following: Omnibus is designed to run with a minimal set of prerequisites. You will need the following:
- Ruby 1.9+ - Ruby 2.0.0+
- Bundler - Bundler
...@@ -274,6 +274,24 @@ omnibus manifest PROJECT -l warn ...@@ -274,6 +274,24 @@ omnibus manifest PROJECT -l warn
This will output a JSON-formatted manifest containing the resolved This will output a JSON-formatted manifest containing the resolved
version of every software definition. version of every software definition.
Whitelisting Libraries
----------------------
Sometimes a platform has libraries that need to be whitelisted so the healthcheck
can pass. The whitelist found in the [healthcheck](https://github.com/chef/omnibus/blob/master/lib/omnibus/health_check.rb)
code comprises the minimal required for successful builds on supported platforms.
To add your own whitelisted library, simply add the a regex to your software
definition in your omnibus project as follows:
```
whitelist_file /libpcrecpp\.so\..+/
```
It is typically a good idea to add a conditional to whitelist based on the specific
platform that requires it.
*Warning: You should only add libraries to the whitelist that are guaranteed to
be on the system you install to; if a library comes from a non-default package
you should instead build it into the package.*
Changelog Changelog
--------- ---------
......
version: "{build}"
os: Windows Server 2012
platform:
- x64
environment:
matrix:
- ruby_version: "200"
clone_folder: c:\projects\omnibus
clone_depth: 1
skip_tags: true
branches:
only:
- master
install:
- SET PATH=C:\Ruby%ruby_version%\bin;C:\Ruby21%ruby_version:~3%\Devkit\mingw\bin;%PATH%
- echo %PATH%
- ruby --version
- gem --version
- gem install bundler --quiet --no-ri --no-rdoc
- bundler --version
- cp C:\Ruby21%ruby_version:~3%\Devkit\mingw\bin\bsdtar.exe C:\Ruby21%ruby_version:~3%\Devkit\mingw\bin\tar.exe
- appveyor DownloadFile http://curl.haxx.se/ca/cacert.pem -FileName C:\cacert.pem
- set SSL_CERT_FILE=C:\cacert.pem
build_script:
- bundle install
test_script:
- bundle exec rake unit
- bundle exec rake functional
- bundle exec rake acceptance
...@@ -22,7 +22,7 @@ repository) is ...@@ -22,7 +22,7 @@ repository) is
customize the location of the cache in the `omnibus.rb` config file customize the location of the cache in the `omnibus.rb` config file
using the key `git_cache_dir`. For example: using the key `git_cache_dir`. For example:
git_cache_dir "/opt/ominbus-caches" git_cache_dir "/opt/omnibus-caches"
## How It Works ## ## How It Works ##
......
Feature: omnibus publish Feature: omnibus publish
Scenario: Overriding publishing platform Scenario: Providing platform mappings file
* I run `omnibus publish artifactory fake * --platform debian` * I have a platform mappings file named "platform_mappings.json"
* I run `omnibus publish artifactory fake * --platform-mappings platform_mappings.json`
* the output should contain: * the output should contain:
""" """
Publishing platform has been overriden to 'debian' Publishing will be performed using provided platform mappings.
"""
Scenario: Overriding publishing platform version
* I run `omnibus publish artifactory fake * --platform-version 7`
* the output should contain:
"""
Publishing platform version has been overriden to '7'
""" """
require 'aruba/api' require 'aruba/api'
Given(/^I have an omnibus project named "(.+)"$/) do |name| Given(/^I have an omnibus project named "(.+)"$/) do |name|
create_dir(name) create_directory(name)
cd(name) cd(name)
write_file("config/projects/#{name}.rb", <<-EOH.gsub(/^ {4}/, '')) write_file("config/projects/#{name}.rb", <<-EOH.gsub(/^ {4}/, ''))
...@@ -27,3 +27,15 @@ Given(/^I have an omnibus project named "(.+)"$/) do |name| ...@@ -27,3 +27,15 @@ Given(/^I have an omnibus project named "(.+)"$/) do |name|
package_tmp './local/omnibus/pkg-tmp' package_tmp './local/omnibus/pkg-tmp'
EOH EOH
end end
Given(/^I have a platform mappings file named "(.+)"$/) do |name|
write_file(name, <<-EOH.gsub(/^ {4}/, ''))
{
"ubuntu-10.04": [
"ubuntu-10.04",
"ubuntu-12.04",
"ubuntu-14.04"
]
}
EOH
end
...@@ -4,10 +4,12 @@ require 'aruba/in_process' ...@@ -4,10 +4,12 @@ require 'aruba/in_process'
require 'omnibus/cli' require 'omnibus/cli'
Aruba.configure do |config|
config.command_launcher = :in_process
config.main_class = Omnibus::CLI::Runner
end
Before do Before do
# Reset anything that might have been cached in the Omnibus project # Reset anything that might have been cached in the Omnibus project
Omnibus.reset!(true) Omnibus.reset!(true)
Aruba::InProcess.main_class = Omnibus::CLI::Runner
Aruba.process = Aruba::InProcess
end end
...@@ -488,10 +488,16 @@ module Omnibus ...@@ -488,10 +488,16 @@ module Omnibus
# @return (see #command) # @return (see #command)
# #
def copy(source, destination, options = {}) def copy(source, destination, options = {})
build_commands << BuildCommand.new("copy `#{source}' to `#{destination}'") do command = "copy `#{source}' to `#{destination}'"
build_commands << BuildCommand.new(command) do
Dir.chdir(software.project_dir) do Dir.chdir(software.project_dir) do
FileSyncer.glob(source).each do |file| files = FileSyncer.glob(source)
FileUtils.cp_r(file, destination, options) if files.empty?
log.warn(log_key) {"no matched files for glob #{command}"}
else
files.each do |file|
FileUtils.cp_r(file, destination, options)
end
end end
end end
end end
...@@ -499,7 +505,7 @@ module Omnibus ...@@ -499,7 +505,7 @@ module Omnibus
expose :copy expose :copy
# #
# Copy the given source to the destination. This method accepts a single # Move the given source to the destination. This method accepts a single
# file or a file pattern to match # file or a file pattern to match
# #
# @param [String] source # @param [String] source
...@@ -511,10 +517,16 @@ module Omnibus ...@@ -511,10 +517,16 @@ module Omnibus
# @return (see #command) # @return (see #command)
# #
def move(source, destination, options = {}) def move(source, destination, options = {})
build_commands << BuildCommand.new("move `#{source}' to `#{destination}'") do command = "move `#{source}' to `#{destination}'"
build_commands << BuildCommand.new(command) do
Dir.chdir(software.project_dir) do Dir.chdir(software.project_dir) do
FileSyncer.glob(source).each do |file| files = FileSyncer.glob(source)
FileUtils.mv(file, destination, options) if files.empty?
log.warn(log_key) {"no matched files for glob #{command}"}
else
files.each do |file|
FileUtils.mv(file, destination, options)
end
end end
end end
end end
...@@ -534,10 +546,16 @@ module Omnibus ...@@ -534,10 +546,16 @@ module Omnibus
# @return (see #command) # @return (see #command)
# #
def link(source, destination, options = {}) def link(source, destination, options = {})
build_commands << BuildCommand.new("link `#{source}' to `#{destination}'") do command = "link `#{source}' to `#{destination}'"
build_commands << BuildCommand.new(command) do
Dir.chdir(software.project_dir) do Dir.chdir(software.project_dir) do
FileSyncer.glob(source).each do |file| files = FileSyncer.glob(source)
FileUtils.ln_s(file, destination, options) if files.empty?
log.warn(log_key) {"no matched files for glob #{command}"}
else
files.each do |file|
FileUtils.ln_s(file, destination, options)
end
end end
end end
end end
...@@ -581,7 +599,7 @@ module Omnibus ...@@ -581,7 +599,7 @@ module Omnibus
def build def build
log.info(log_key) { 'Starting build' } log.info(log_key) { 'Starting build' }
shasum # ensure shashum is calculated before build since the build can alter the shasum shasum # ensure shashum is calculated before build since the build can alter the shasum
log.internal(log_key) { 'Cached builder checksum before build: #{shasum}'} log.internal(log_key) { "Cached builder checksum before build: #{shasum}"}
if software.overridden? if software.overridden?
log.info(log_key) do log.info(log_key) do
"Version overridden from #{software.default_version} to "\ "Version overridden from #{software.default_version} to "\
......
...@@ -85,7 +85,7 @@ module Omnibus ...@@ -85,7 +85,7 @@ module Omnibus
if @options[:output_manifest] if @options[:output_manifest]
FileUtils.mkdir_p('pkg') FileUtils.mkdir_p('pkg')
File.open(::File.join('pkg', 'version-manifest.json'), 'w') do |f| File.open(::File.join('pkg', 'version-manifest.json'), 'w') do |f|
f.write(JSON.pretty_generate(project.built_manifest.to_hash)) f.write(project.built_manifest.to_json)
end end
end end
end end
......
...@@ -66,37 +66,84 @@ module Omnibus ...@@ -66,37 +66,84 @@ module Omnibus
desc: "Explicit version for this changelog", desc: "Explicit version for this changelog",
type: :string type: :string
desc 'generate', 'Generate a changelog for a new release' desc 'generate [START] [END]', 'Generate a changelog for a new release'
def generate def generate(start_ref=nil, end_ref=nil)
g = GitRepository.new @start_ref = start_ref
@end_ref = end_ref
if @options[:skip_components] diff = if @options[:skip_components]
diff = Omnibus::EmptyManifestDiff.new Omnibus::EmptyManifestDiff.new
else else
old_manifest = if @options[:starting_manifest] Omnibus::ManifestDiff.new(old_manifest, new_manifest)
Omnibus::Manifest.from_file(@options[:starting_manifest]) end
else
Omnibus::Manifest.from_hash(JSON.parse(g.file_at_revision("version-manifest.json",
g.latest_tag)))
end
new_manifest = Omnibus::Manifest.from_file(@options[:ending_manifest])
diff = Omnibus::ManifestDiff.new(old_manifest, new_manifest)
end
new_version = if @options[:version]
@options[:version]
elsif @options[:patch]
Omnibus::SemanticVersion.new(g.latest_tag).next_patch.to_s
elsif @options[:minor] && !@options[:major] # minor is the default so it will always be true
Omnibus::SemanticVersion.new(g.latest_tag).next_minor.to_s
elsif @options[:major]
Omnibus::SemanticVersion.new(g.latest_tag).next_major.to_s
end
Omnibus::ChangeLogPrinter.new(ChangeLog.new(), Omnibus::ChangeLogPrinter.new(ChangeLog.new(starting_revision, ending_revision),
diff, diff,
@options[:source_path]).print(new_version) @options[:source_path]).print(new_version)
end end
private
def local_git_repo
GitRepository.new
end
def old_manifest
@old_manifest ||= if @options[:starting_manifest]
Omnibus::Manifest.from_file(@options[:starting_manifest])
else
manifest_for_revision(starting_revision)
end
end
def new_manifest
@new_manifest ||= if @options[:ending_manifest]
Omnibus::Manifest.from_file(@options[:ending_manifest])
else
manifest_for_revision(ending_revision)
end
end
def manifest_for_revision(rev)
Omnibus::Manifest.from_hash(JSON.parse(local_git_repo.file_at_revision("version-manifest.json", rev)))
end
def new_version
if @options[:version]
@options[:version]
elsif @options[:patch]
Omnibus::SemanticVersion.new(local_git_repo.latest_tag).next_patch.to_s
elsif @options[:minor] && !@options[:major] # minor is the default so it will always be true
Omnibus::SemanticVersion.new(local_git_repo.latest_tag).next_minor.to_s
elsif @options[:major]
Omnibus::SemanticVersion.new(local_git_repo.latest_tag).next_major.to_s
elsif @options[:ending_manifest]
new_manifest.build_version
end
end
# starting_revision is taken from:
# - value passed as the first argument
# - value found in the starting manifest
# - the latest git tag in the local repository
def starting_revision
@start_ref ||= if @options[:starting_manifest]
old_manifest.build_git_revision
else
local_git_repo.latest_tag
end
end
# ending_revision is taken from:
# - value passed as the first argument
# - value found in the ending manifest
# - HEAD in the current git repository
def ending_revision
@end_ref ||= if @options[:ending_manifest]
new_manifest.build_git_revision
else
'HEAD'
end
end
end end
end end
...@@ -18,19 +18,27 @@ module Omnibus ...@@ -18,19 +18,27 @@ module Omnibus
class Command::Publish < Command::Base class Command::Publish < Command::Base
namespace :publish namespace :publish
# These options are useful for publish packages that were built for a # This option is useful for publish packages that were built for a
# paticluar platform/version and tested on another platform/version. # particular platform/version but tested on other platform/versions.
# #
# For example, one might build on Ubuntu 10.04 and test/publish on # For example, one might build on Ubuntu 10.04 and test/publish on
# Ubuntu 11.04, 12.04 and Debian 7. # Ubuntu 10.04, 12.04, and 14.04.
# #
# If these options are used with the glob pattern support all packages # @example JSON
# will be published to the same platform/version. # {
class_option :platform, # "ubuntu-10.04": [
desc: 'The platform to publish for', # "ubuntu-10.04",
# "ubuntu-12.04",
# "ubuntu-14.04"
# ]
# }
#
class_option :platform_mappings,
desc: 'The optional platform mappings JSON file to publish with',
type: :string type: :string
class_option :platform_version,
desc: 'The platform version to publish for', class_option :version_manifest,
desc: 'Path to the version-manifest.json file to publish with',
type: :string type: :string
# #
...@@ -43,6 +51,10 @@ module Omnibus ...@@ -43,6 +51,10 @@ module Omnibus
desc: 'The accessibility of the uploaded packages', desc: 'The accessibility of the uploaded packages',
enum: %w(public private), enum: %w(public private),
default: 'private' default: 'private'
method_option :region,
type: :string,
desc: 'The region in which the bucket is located',
default: 'us-east-1'
desc 's3 BUCKET PATTERN', 'Publish to an S3 bucket' desc 's3 BUCKET PATTERN', 'Publish to an S3 bucket'
def s3(bucket, pattern) def s3(bucket, pattern)
options[:bucket] = bucket options[:bucket] = bucket
...@@ -54,6 +66,10 @@ module Omnibus ...@@ -54,6 +66,10 @@ module Omnibus
# #
# $ omnibus publish artifactory libs-omnibus-local pkg/chef* # $ omnibus publish artifactory libs-omnibus-local pkg/chef*
# #
method_option :build_record,
type: :boolean,
desc: 'Optionally create an Artifactory build record for the published artifacts',
default: true
desc 'artifactory REPOSITORY PATTERN', 'Publish to an Artifactory instance' desc 'artifactory REPOSITORY PATTERN', 'Publish to an Artifactory instance'
def artifactory(repository, pattern) def artifactory(repository, pattern)
options[:repository] = repository options[:repository] = repository
...@@ -68,8 +84,12 @@ module Omnibus ...@@ -68,8 +84,12 @@ module Omnibus
# @return [void] # @return [void]
# #
def publish(klass, pattern, options) def publish(klass, pattern, options)
if options[:platform_mappings]
options[:platform_mappings] = JSON.parse(File.read(File.expand_path(options[:platform_mappings])))
end
klass.publish(pattern, options) do |package| klass.publish(pattern, options) do |package|
say("Uploaded '#{package.name}'", :green) say("Published '#{package.name}' for #{package.metadata[:platform]}-#{package.metadata[:platform_version]}", :green)
end end
end end
end end
......
...@@ -289,6 +289,14 @@ module Omnibus ...@@ -289,6 +289,14 @@ module Omnibus
raise MissingRequiredAttribute.new(self, :s3_secret_key, "'EFGH5678'") raise MissingRequiredAttribute.new(self, :s3_secret_key, "'EFGH5678'")
end end
# The region of the S3 bucket you want to cache software artifacts in.
# Defaults to 'us-east-1'
#
# @return [String]
default(:s3_region) do
'us-east-1'
end
# -------------------------------------------------- # --------------------------------------------------
# @!endgroup # @!endgroup
# #
...@@ -443,6 +451,26 @@ module Omnibus ...@@ -443,6 +451,26 @@ module Omnibus