Merge pull request #94 from kleetus/node-pre-gyp

Binary distributions

.travis.yml

@@ -12,5 +12,4 @@ script:
   - _mocha -R spec --recursive
 cache:
   directories:
-    - platform/ubuntu
-
+    - cache 

Makefile

@@ -1,9 +0,0 @@
-all:
-	@node-gyp clean 2>/dev/null
-	node-gyp -d configure
-	node-gyp build
-
-clean:
-	@node-gyp clean 2>/dev/null
-
-.PHONY: all clean

README.md

@@ -1,7 +1,7 @@
 Bitcore Node
 =======
 
-A Node.js module that adds a native interface to Bitcoin Core for querying information about the Bitcoin blockchain. Bindings are linked to Bitcoin Core compiled as a shared library.
+A Node.js module that adds a native interface to Bitcoin Core for querying information about the Bitcoin blockchain. Bindings are linked to Bitcoin Core compiled as a static library.
 
 ## Install
 
@@ -35,7 +35,7 @@ nvm install v0.12
 To build Bitcoin Core and bindings development packages are needed:
 
 ```bash
-sudo apt-get install build-essential libtool autotools-dev autoconf pkg-config libssl-dev
+sudo apt-get install build-essential libtool autotools-dev automake autoconf pkg-config libssl-dev
 ```
 
 Clone the bitcore-node repository locally:
@@ -45,7 +45,7 @@ git clone https://github.com/bitpay/bitcore-node.git
 cd bitcore-node
 ```
 
-And finally run the build which will take several minutes. A script in the "bin" directory will download Bitcoin Core v0.11, apply a shared library patch (see more info below), and compile the shared library and Node.js bindings, and then copy build artifacts and header files into `platform/ubuntu`. You can start this by running:
+And finally run the build which will take several minutes. A script in the "bin" directory will download Bitcoin Core v0.11, apply a patch (see more info below), and compile the static library and Node.js bindings. You can start this by running:
 
 ```bash
 npm install
@@ -86,7 +86,7 @@ git clone https://github.com/bitpay/bitcore-node.git
 cd bitcore-node
 ```
 
-And finally run the build which will take several minutes. A script in the "bin" directory will download Bitcoin Core v0.11, apply a shared library patch (see more info below), and compile the shared library and Node.js bindings, and then copy build artifacts and header files into `platform/osx`. You can start this by running:
+And finally run the build which will take several minutes. A script in the "bin" directory will download Bitcoin Core v0.11, apply a patch (see more info below), and compile the static library and Node.js bindings. You can start this by running:
 
 ```bash
 npm install
@@ -111,8 +111,7 @@ To run tests against the bindings, as defined in `bindings.gyp` the regtest feat
 
 ```bash
 export BITCORENODE_ENV=test
-rm -rf platform/<os_name>/*
-npm install
+npm run build
 ```
 
 If you do not already have mocha installed:
@@ -133,6 +132,8 @@ If any changes have been made to the bindings in the "src" directory, manually c
 $ node-gyp -d rebuild
 ```
 
+Note: `node-gyp` can be installed with `npm install node-gyp -g`
+
 To be able to debug you'll need to have `gdb` and `node` compiled for debugging with gdb using `--gdb` (sometimes called node_g), and you can then run:
 
 ```bash
@@ -151,9 +152,9 @@ $ cd benchmarks
 $ node index.js
 ```
 
-## Shared Library Patch
+## Static Library Patch
 
-To provide native bindings to JavaScript *(or any other language for that matter)*, Bitcoin code, itself, must be linkable. Currently, Bitcoin Core provides a JSON RPC interface to bitcoind as well as a shared library for script validation *(and hopefully more)* called libbitcoinconsensus. There is a node module, [node-libbitcoinconsensus](https://github.com/bitpay/node-libbitcoinconsensus), that exposes these methods. While these interfaces are useful for several use cases, there are additional use cases that are not fulfilled, and being able to implement customized interfaces is necessary. To be able to do this a few simple changes need to be made to Bitcoin Core to compile as a shared library.
+To provide native bindings to JavaScript *(or any other language for that matter)*, Bitcoin code, itself, must be linkable. Currently, Bitcoin Core provides a JSON RPC interface to bitcoind as well as a shared library for script validation *(and hopefully more)* called libbitcoinconsensus. There is a node module, [node-libbitcoinconsensus](https://github.com/bitpay/node-libbitcoinconsensus), that exposes these methods. While these interfaces are useful for several use cases, there are additional use cases that are not fulfilled, and being able to implement customized interfaces is necessary. To be able to do this a few simple changes need to be made to Bitcoin Core to compile as a static library.
 
 The patch is located at `etc/bitcoin.patch` and adds a configure option `--enable-daemonlib` to compile all object files with `-fPIC` (Position Independent Code - needed to create a shared object), exposes leveldb variables and objects, exposes the threadpool to the bindings, and conditionally includes the main function.
 

RELEASE.md

@@ -0,0 +1,34 @@
+## Release Process
+
+Binaries for the C++ binding file (which includes libbitcoind statically linked in) are distributed for convenience. The binary binding file `bitcoind.node` is published to S3 for later download and installation. Source files can also be built if binaries are not desired.
+
+### How to Release
+
+Ensure you've followed the instructions in the README.md for building the project from source. You will be using node-gyp to build the C++ bindings. A script will then upload the bindings to S3 for later use. You will also need credentials for BitPay's bitcore-node S3 bucket and be listed as an author for the bitcore-node's npm module.
+
+- Create a file `.bitcore-node-upload.json` in your home directory
+- The format of this file should be:
+
+```json
+{
+  "region": "us-east-1",
+  "accessKeyId": "xxx",
+  "secretAccessKey": "yyy"
+}
+```
+
+To make a release, bump the version of the package.json:
+
+```bash
+git commit -a -m "Bump package version to <version>"
+npm install
+npm run upload
+npm publish
+```
+
+And then update the version of the package.json for development (e.g. "0.3.2-dev"):
+
+```bash
+git commit -a -m "Bump development version to <version>"
+git push upstream master
+```

bin/build

@@ -0,0 +1,163 @@
+#!/bin/bash
+root_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/.."
+options=`cat ${root_dir}/bin/config_options.sh`
+depends_dir=$($root_dir/bin/variables.sh depends_dir)
+host=$(${root_dir}/bin/variables.sh host)
+btc_dir="${root_dir}/libbitcoind"
+patch_sha=$($root_dir/bin/variables.sh patch_sha)
+export CPPFLAGS="-I${depends_dir}/${host}/include/boost -I${depends_dir}/${host}/include -L${depends_dir}/${host}/lib" 
+echo "Using BTC directory: ${btc_dir}"
+
+cd "${root_dir}"
+
+build_dependencies () {
+  if [ -d "${btc_dir}" ]; then
+    pushd "${depends_dir}"
+    echo "using host for dependencies: ${host}"
+    if [ "${test}" = true ]; then
+      make HOST=${host} NO_QT=1 NO_UPNP=1
+    else
+      make HOST=${host} NO_QT=1 NO_WALLET=1 NO_UPNP=1
+    fi
+    popd
+  fi 
+}
+
+get_patch_file () {
+  if test -e "${root_dir/PATCH_VERSION}"; then
+    tag=`cat "${root_dir}/PATCH_VERSION" | xargs`
+  else
+    echo "no tag file found, please create it in the root of the project as so: 'echo \"v0.10.2\" > PATCH_VERSION'"
+    exit 1
+  fi
+}
+
+compare_patch () {
+  cd "${btc_dir}"
+  get_patch_file
+  echo "running the diff command from HEAD to ${tag}"
+  git diff ${tag}..HEAD > /tmp/tmp.patch
+  matching_patch=`diff -w /tmp/tmp.patch "${root_dir}/etc/bitcoin.patch"` 
+}
+
+cache_files () {
+  cache_file="${root_dir}"/cache/cache.tar
+  pushd "${btc_dir}"
+  find . -type f \( -name "*.h" -or -name "*.hpp" -or -name \
+"*.ipp" -or -name "*.a" \) | tar -cf "${cache_file}" -T -
+  tar xf "${cache_file}" -C "${root_dir}"/cache
+  rm -fr "${cache_file}" 
+  popd
+}
+
+debug=
+if [ "${BITCORENODE_ENV}" == "debug" ]; then
+  options=`cat ${root_dir}/bin/config_options_debug.sh`
+fi
+
+test=false
+if [ "${BITCORENODE_ENV}" == "test" ]; then
+  test=true
+  options=`cat ${root_dir}/bin/config_options_test.sh`
+fi
+
+patch_file_sha=$(shasum -a 256 "${root_dir}/etc/bitcoin.patch" | awk '{print $1}')
+last_patch_file_sha=
+if [ -e "${patch_sha}" ]; then 
+  echo "Patch file sha exists, let's see if the patch has changed since last build..."
+  last_patch_file_sha=$(cat "${patch_sha}")
+fi
+shared_file_built=false
+if [ "${last_patch_file_sha}" == "${patch_file_sha}" ]; then
+  echo "Patch file contents matches the sha from the patch file itself, so no reason to rebuild the bindings unless there are no prebuilt bindings."
+  shared_file_built=true
+fi
+
+if [ "${shared_file_built}" = false ]; then
+  echo "Looks like the patch to bitcoin changed since last build -or- this is the first build, so rebuilding libbitcoind itself..."
+  mac_response=$($root_dir/bin/variables.sh mac_dependencies) 
+  if [ "${mac_response}" != "" ]; then
+    echo "${mac_response}" 
+    exit -1
+  fi
+  only_make=false
+  if [ -d "${btc_dir}" ]; then
+    echo "running compare patch..."
+    compare_patch
+    repatch=false
+    if [[ "${matching_patch}" =~ [^\s\\] ]]; then
+      echo "Warning! libbitcoind is not patched with:\
+        ${root_dir}/etc/bitcoin.patch."
+      echo -n "Would you like to remove the current patch, checkout the tag: ${tag} and \
+apply the current patch from "${root_dir}"/etc/bitcoin.patch? (y/N): "
+      if [ "${BITCORENODE_ASSUME_YES}" = true ]; then
+        input=y
+        echo ""
+      else
+        read input
+      fi
+      if [[ "${input}" =~ ^y|^Y ]]; then
+        repatch=true
+        echo "Removing directory: \"${btc_dir}\" and starting over!"
+        rm -fr "${btc_dir}"
+      fi
+    fi
+    if [ "${repatch}" = false ]; then
+      echo "Running make inside libbitcoind (assuming you've previously patched and configured libbitcoind)..."
+      cd "${btc_dir}"
+      only_make=true
+    fi
+  fi
+
+  if [ "${only_make}" = false ]; then
+    echo "Cloning, patching, and building libbitcoind..."
+    get_patch_file
+    echo "attempting to checkout tag: ${tag} of bitcoin from github..."
+    cd "${root_dir}"
+    git clone --depth 1 --branch "${tag}" https://github.com/bitcoin/bitcoin.git libbitcoind
+
+    cd "${btc_dir}"
+
+    echo '../patch-bitcoin.sh' "${btc_dir}"
+    ../bin/patch-bitcoin "${btc_dir}"
+
+    if ! test -d .git; then
+      echo 'Please point this script to an upstream bitcoin git repo.'
+      exit 1
+    fi
+
+  fi
+  build_dependencies
+  echo './autogen.sh'
+  ./autogen.sh
+
+  boost_libdir="--with-boost-libdir=${depends_dir}/${host}/lib"
+
+  full_options="${options} ${boost_libdir}"
+  echo "running the configure script with the following options:\n :::[\"${full_options}\"]:::"
+  ${full_options}
+
+  echo 'make V=1'
+  make V=1
+
+  echo "Creating the sha marker for the patching in libbitcoind..."
+  echo "Writing patch sha file to: \"${patch_sha}\""
+  echo -n `shasum -a 256 "${root_dir}"/etc/bitcoin.patch | awk '{print $1}'` > "${patch_sha}"
+  cache_files 
+  echo 'Build finished successfully.'
+else
+  echo 'Using existing static library.'
+fi
+
+# Building the Bindings
+
+set -e
+
+cd "${root_dir}"
+
+debug=--debug=false
+if test x"$1" = x'debug'; then
+  debug=--debug
+fi
+
+node-gyp ${debug} rebuild

bin/build-bindings

@@ -1,14 +0,0 @@
-#!/bin/bash
-set -e
-
-root_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/.."
-cd "${root_dir}"
-
-debug=
-if test x"$1" = x'debug'; then
-  debug=--debug
-fi
-
-node-gyp ${debug} rebuild
-
-

bin/build-libbitcoind

@@ -1,186 +0,0 @@
-#!/bin/bash
-root_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/.."
-options=`cat ${root_dir}/bin/config_options.sh`
-h_and_a_dir=$($root_dir/platform/os.sh h_and_a_dir)
-depends_dir=$($root_dir/platform/os.sh depends_dir)
-os_dir=$(${root_dir}/platform/os.sh osdir)
-host=$(${root_dir}/platform/os.sh host)
-btc_dir="${root_dir}/libbitcoind"
-ext=$($root_dir/platform/os.sh ext)
-artifacts_dir=$($root_dir/platform/os.sh artifacts_dir)
-export CPPFLAGS="-I${h_and_a_dir}/include/boost -I${h_and_a_dir}/include -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/include -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.10.sdk/usr/include -L${h_and_a_dir}/lib"
-echo "Using BTC directory: ${btc_dir}"
-
-cd "${root_dir}"
-
-copy_header_files () {
-  if [[ -d "${artifacts_dir}" && -d "${h_and_a_dir}" && -d "${btc_dir}" ]]; then
-    mkdir -p "${artifacts_dir}/include" > /dev/null 2>&1
-    pushd "${root_dir}"
-    echo "Copying headers for caching purposes, this can take a while (but will only be done once), please wait..."
-    find libbitcoind -type f \( -name "*.h" -or -name "*.hpp" -or -name "*.ipp" \) -print0 | xargs -0 -I{} rsync -R {} "${artifacts_dir}"/include
-    mkdir -p "${artifacts_dir}/lib" > /dev/null 2>&1
-    cp -r "${h_and_a_dir}"/lib/libboost_filesystem-mt.a "${h_and_a_dir}"/lib/libboost_thread-mt.a "${artifacts_dir}"/lib/ 
-    popd
-  fi
-}
-
-build_dependencies () {
-  if [ -d "${btc_dir}" ]; then
-    pushd "${depends_dir}"
-    echo "using host for dependencies: ${host}"
-    boost_files_count=`find "${h_and_a_dir}"/lib -iname "libboost_*-mt.a" | wc -l | xargs`
-    db_files_count=`find "${h_and_a_dir}"/lib -iname "libdb*.a" | wc -l | xargs`
-    should_rebuild=false
-    if [[ "${boost_files_count}" -lt 5 || ( "${db_files_count}" -lt 1  && "${test}" = true ) ]]; then
-      should_rebuild=true
-    fi
-    if [ "${should_rebuild}" = true ]; then
-      if [ "${test}" = true ]; then
-        make HOST=${host} NO_QT=1 NO_UPNP=1
-      else
-        make HOST=${host} NO_QT=1 NO_WALLET=1 NO_UPNP=1
-      fi
-    else
-      echo "Looks like libs are already built, so we won't rebuild them. Incidentally, we found: ${boost_files_count} boost libraries and: ${db_files_count} Berkeley DB libraries and you have test mode set to: ${test}"
-    fi
-    popd
-  fi 
-}
-
-get_patch_file () {
-  if test -e "${root_dir/PATCH_VERSION}"; then
-    tag=`cat "${root_dir}/PATCH_VERSION" | xargs`
-  else
-    echo "no tag file found, please create it in the root of the project as so: 'echo \"v0.10.2\" > PATCH_VERSION'"
-    exit 1
-  fi
-}
-
-compare_patch () {
-  cd "${btc_dir}"
-  get_patch_file
-  echo "running the diff command from HEAD to ${tag}"
-  git diff ${tag}..HEAD > /tmp/tmp.patch
-  matching_patch=`diff -w /tmp/tmp.patch "${root_dir}/etc/bitcoin.patch"` 
-}
-
-debug=
-if [ "${BITCORENODE_ENV}" == "debug" ]; then
-  options=`cat ${root_dir}/bin/config_options_debug.sh`
-fi
-
-test=false
-if [ "${BITCORENODE_ENV}" == "test" ]; then
-  test=true
-  options=`cat ${root_dir}/bin/config_options_test.sh`
-fi
-
-patch_file_sha=$(shasum -a 256 "${root_dir}/etc/bitcoin.patch" | awk '{print $1}')
-patch_sha=`find "${os_dir}" -iname patch_sha.txt`
-last_patch_file_sha=
-if [ -e "${patch_sha}" ]; then 
-  if [ "${ext}" == "dylib" ]; then
-    last_patch_file_sha=$(cat "${os_dir}"/lib/patch_sha.txt)
-  else
-    last_patch_file_sha=$(cat "${os_dir}"/patch_sha.txt)
-  fi
-fi
-shared_file_built=false
-if [[ "${last_patch_file_sha}" == "${patch_file_sha}" && -d "${artifacts_dir}/include" ]]; then
-  shared_file_built=true
-fi
-
-if [ "${shared_file_built}" = false ]; then
-  mac_response=$($root_dir/platform/os.sh mac_dependencies) 
-  if [ "${mac_response}" != "" ]; then
-    echo "${mac_response}" 
-    exit -1
-  fi
-  only_make=false
-  if [ -d "${btc_dir}" ]; then
-    echo "running compare patch..."
-    compare_patch
-    repatch=false
-    if [[ "${matching_patch}" =~ [^\s\\] ]]; then
-      echo "Warning! libbitcoind is not patched with:\
-        ${root_dir}/etc/bitcoin.patch."
-      echo -n "Would you like to remove the current patch, checkout the tag: ${tag} and \
-apply the current patch from "${root_dir}"/etc/bitcoin.patch? (y/N): "
-      if [ "${BITCORENODE_ASSUME_YES}" = true ]; then
-        input=y
-        echo ""
-      else
-        read input
-      fi
-      if [[ "${input}" =~ ^y|^Y ]]; then
-        repatch=true
-        rm -f "${os_dir}/libbitcoind.*"
-        echo "Removing directory: \"${btc_dir}\" and starting over!"
-        rm -fr "${btc_dir}"
-      fi
-    fi
-    if [ "${repatch}" = false ]; then
-      echo "Running make inside libbitcoind (assuming you've previously patched and configured libbitcoind)..."
-      cd "${btc_dir}"
-      only_make=true
-    fi
-  fi
-
-  if [ "${only_make}" = false ]; then
-    echo "Cloning, patching, and building libbitcoind..."
-    get_patch_file
-    echo "attempting to checkout tag: ${tag} of bitcoin from github..."
-    cd "${root_dir}"
-    git clone --depth 1 --branch "${tag}" https://github.com/bitcoin/bitcoin.git libbitcoind
-
-    cd "${btc_dir}"
-
-    echo '../patch-bitcoin.sh' "${btc_dir}"
-    ../bin/patch-bitcoin "${btc_dir}"
-
-    if ! test -d .git; then
-      echo 'Please point this script to an upstream bitcoin git repo.'
-      exit 1
-    fi
-
-  fi
-  build_dependencies
-  echo './autogen.sh'
-  ./autogen.sh
-
-  boost_libdir="--with-boost-libdir=${h_and_a_dir}/lib"
-
-  full_options="${options} ${boost_libdir} --prefix=${os_dir}"
-  echo "running the configure script with the following options:\n :::[\"${full_options}\"]:::"
-  ${full_options}
-
-  echo 'make V=1'
-  make V=1
-
-  echo 'Copying libbitcoind.{so|dylib} to its appropriate location.'
-  if test -e "${btc_dir}/src/.libs/libbitcoind.${ext}"; then
-    if [ "$ext" = "dylib" ]; then
-      if [ ! -d "${os_dir}/lib" ]; then
-        mkdir -p "${os_dir}/lib"
-      fi
-      cp -R "${btc_dir}"/src/.libs/libbitcoind.*dylib "${os_dir}/lib/"
-    else
-      if [ ! -d "${os_dir}" ]; then
-        mkdir -p "${os_dir}"
-      fi
-      cp -P "${btc_dir}"/src/.libs/libbitcoind.so* "${os_dir}/"
-    fi
-    echo "Creating the sha marker for the patching in libbitcoind..."
-    echo -n `shasum -a 256 "${root_dir}"/etc/bitcoin.patch | awk '{print $1}'` > "${artifacts_dir}"/patch_sha.txt
-    echo "Copying the header files in order to be cached..."
-    copy_header_files
-  else
-    echo "Could not find the shared libraries after they should have been built, please run make clean inside the libbitcoind dir and run npm install again."
-    exit 1
-  fi
-  echo 'Build finished successfully.'
-else
-  echo 'Using existing shared library.'
-fi
-

bin/clean

@@ -0,0 +1,9 @@
+#!/bin/bash
+
+root_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/.."
+cd "${root_dir}"
+pushd "${root_dir}"/libbitcoind
+make clean
+popd
+node-gyp clean
+rm -fr cache/*

bin/get-tarball-name.js

@@ -0,0 +1,11 @@
+'use strict';
+
+var packageRoot = __dirname + '/..';
+var version = require(packageRoot + '/package.json').version;
+var platform = process.platform;
+var arch = process.arch;
+var tarballName = 'libbitcoind-' + version + '-' + platform + '-' + arch + '.tgz';
+
+if (require.main === module) {
+  process.stdout.write(tarballName);
+}

bin/install

@@ -0,0 +1,26 @@
+#!/bin/bash
+
+root_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/.."
+
+cd "${root_dir}"
+
+tarball_name=`node bin/get-tarball-name.js`
+bucket_name="bitcore-node"
+binary_url="https://${bucket_name}.s3.amazonaws.com/${tarball_name}"
+
+echo "Downloading binary: ${binary_url}"
+
+curl --fail -I $binary_url
+
+if test $? -eq 0; then
+  curl $binary_url > $tarball_name
+  if test -e "${tarball_name}"; then
+    echo "Unpacking binary distribution"
+    tar -xvzf $tarball_name
+    if test $? -eq 0; then
+      exit 0
+    fi
+  fi
+fi
+echo "Prebuild binary could not be downloaded, building from source..."
+./bin/build

bin/upload.js

@@ -0,0 +1,77 @@
+'use strict';
+
+var fs = require('fs');
+var chainlib = require('chainlib');
+var log = chainlib.log;
+var AWS = require('aws-sdk');
+
+var config = require(process.env.HOME + '/.bitcore-node-upload.json');
+
+AWS.config.region = config.region;
+AWS.config.update({
+  accessKeyId: config.accessKeyId,
+  secretAccessKey: config.secretAccessKey
+});
+
+var bindings = require('bindings');
+var packageRoot = bindings.getRoot(bindings.getFileName());
+var binaryPath = bindings({
+  path: true,
+  bindings: 'bitcoind.node'
+});
+
+var relativeBinaryPath = binaryPath.replace(packageRoot + '/', '');
+var exec = require('child_process').exec;
+var version = require(packageRoot + '/package.json').version;
+var platform = process.platform;
+var arch = process.arch;
+var tarballName = 'libbitcoind-' + version + '-' + platform + '-' + arch + '.tgz';
+var bucketName = 'bitcore-node';
+var url = 'https://' + bucketName + '.s3.amazonaws.com/' + tarballName;
+
+var child = exec('tar -C ' + packageRoot + ' -cvzf ' + tarballName + ' ' + relativeBinaryPath,
+  function (error, stdout, stderr) {
+
+    if (error) {
+      throw error;
+    }
+
+    if (stdout) {
+      log.info('Tar:', stdout);
+    }
+
+    if (stderr) {
+      log.error(stderr);
+    }
+
+    var fileStream = fs.createReadStream(packageRoot + '/' + tarballName);
+
+    fileStream.on('error', function(err) {
+      if (err) {
+        throw err;
+      }
+    });
+
+    fileStream.on('open', function() {
+
+      var s3 = new AWS.S3();
+
+      var params = {
+        ACL: 'public-read',
+        Key: tarballName,
+        Body: fileStream,
+        Bucket: bucketName
+      };
+
+       s3.putObject(params, function(err, data) {
+         if (err) {
+           throw err;
+         } else {
+           log.info('Successfully uploaded to: ' + url);
+         }
+       });
+
+    });
+
+  }
+);

bin/variables.sh

@@ -0,0 +1,115 @@
+#!/bin/bash
+
+exec 2> /dev/null
+
+root_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/.."
+bitcoin_dir="${root_dir}"/libbitcoind
+cache_dir="${root_dir}"/cache
+
+platform=`uname -a | awk '{print tolower($1)}'`
+arch=`uname -m`
+host="${arch}"-"${platform}"
+
+mac_response=
+check_mac_build_system () {
+  if [ "${platform}" == "darwin" ]; then
+    if [ ! -d "/usr/include" ]; then
+      if hash xcode-select 2>/dev/null; then
+        mac_response="Please run 'xcode-select --install' from the command line because it seems that you've got Xcode, but not the Xcode command line tools that are required for compiling this project from source..."
+      else
+        mac_response="please use the App Store to install Xcode and Xcode command line tools. After Xcode is installed, please run: 'xcode-select --install' from the command line"
+      fi
+    fi
+  fi
+}
+
+depends_dir="${bitcoin_dir}"/depends
+thread="${cache_dir}"/depends/"${host}"/lib/libboost_thread-mt.a
+filesystem="${cache_dir}"/depends/"${host}"/lib/libboost_filesystem-mt.a
+chrono="${cache_dir}"/depends/"${host}"/lib/libboost_chrono-mt.a
+program_options="${cache_dir}"/depends/"${host}"/lib/libboost_program_options-mt.a
+system="${cache_dir}"/depends/"${host}"/lib/libboost_system-mt.a
+leveldb="${cache_dir}"/src/leveldb/libleveldb.a
+memenv="${cache_dir}"/src/leveldb/libmemenv.a
+libsecp256k1="${cache_dir}"/src/secp256k1/.libs/libsecp256k1.a
+
+if test x"$1" = x'anl'; then
+  if [ "${platform}" != "darwin" ]; then
+    echo -n "-lanl"
+  fi
+fi
+
+if test x"$1" = x'cache_dir'; then
+  echo -n "${cache_dir}"
+fi
+
+if test x"$1" = x'btcdir'; then
+  echo -n "${bitcoin_dir}"
+fi
+
+if test -z "$1" -o x"$1" = x'thread'; then
+  echo -n "${thread}"
+fi
+
+if test -z "$1" -o x"$1" = x'filesystem'; then
+  echo -n "${filesystem}"
+fi
+
+if test -z "$1" -o x"$1" = x'program_options'; then
+  echo -n "${program_options}"
+fi
+
+if test -z "$1" -o x"$1" = x'system'; then
+  echo -n "${system}"
+fi
+
+if test -z "$1" -o x"$1" = x'chrono'; then
+  echo -n "${chrono}"
+fi
+
+if test -z "$1" -o x"$1" = x'depends_dir'; then
+  echo -n "${depends_dir}"
+fi
+
+if test -z "$1" -o x"$1" = x'leveldb'; then
+  echo -n "${leveldb}"
+fi
+
+if test -z "$1" -o x"$1" = x'memenv'; then
+  echo -n "${memenv}"
+fi
+
+if test -z "$1" -o x"$1" = x'libsecp256k1'; then
+  echo -n "${libsecp256k1}"
+fi
+
+if test -z "$1" -o x"$1" = x'host'; then
+  echo -n "${host}"
+fi
+
+if test -z "$1" -o x"$1" = x'bdb'; then
+  if [ "${BITCORENODE_ENV}" == "test" ]; then
+    echo -n "${cache_dir}"/depends/"${host}"/lib/libdb_cxx.a
+  fi
+fi
+
+if test -z "$1" -o x"$1" = x'patch_sha'; then
+  echo -n "${root_dir}"/cache/patch_sha.txt
+fi
+
+if test -z "$1" -o x"$1" = x'load_archive'; then
+  if [ "${os}"  == "osx" ]; then
+    echo -n "-Wl,-all_load -Wl,--no-undefined"
+  else
+    echo -n "-Wl,--whole-archive ${filesystem} ${thread} "${cache_dir}"/src/.libs/libbitcoind.a -Wl,--no-whole-archive"
+  fi
+fi
+
+if test -z "$1" -o x"$1" = x'mac_dependencies'; then
+  check_mac_build_system
+  echo -n "${mac_response}"
+fi
+
+if test -z "$1" -o x"$1" = x'bitcoind'; then
+  echo -n "${cache_dir}"/src/.libs/libbitcoind.a
+fi

binding.gyp

@@ -1,40 +1,53 @@
 {
-  'targets': [{
-    'target_name': 'libbitcoind',
-    'include_dirs' : [
-      '<!(node -e "require(\'nan\')")',
-      '<!(./platform/os.sh artifacts_dir)/include/libbitcoind/src',
-      '<!(./platform/os.sh artifacts_dir)/include/libbitcoind/depends/<!(./platform/os.sh host)/include',
-      '<!(./platform/os.sh artifacts_dir)/include/libbitcoind/src/leveldb/include'
-    ],
-    'sources': [
-      './src/libbitcoind.cc',
-    ],
-    'conditions': [
-        ['OS=="mac"', {
-          'xcode_settings': {
-            'GCC_ENABLE_CPP_EXCEPTIONS': 'YES',
-            'GCC_ENABLE_CPP_RTTI': 'YES',
-            'MACOSX_DEPLOYMENT_TARGET': '10.9'
-          }
-        }
-      ]
-    ],
-    'cflags_cc': [
-      '-fexceptions',
-      '-frtti',
-      '-fpermissive',
-    ],
-    'link_settings': {
-      'libraries': [
-        '<!(./platform/os.sh filesystem)',
-        '<!(./platform/os.sh thread)',
-        '<!(./platform/os.sh lib)'
+  "targets": [
+    {
+      "target_name": "libbitcoind",
+      "include_dirs" : [
+        "<!(node -e \"require('nan')\")",
+        "<!(./bin/variables.sh cache_dir)/src",
+        "<!(./bin/variables.sh cache_dir)/depends/<!(./bin/variables.sh host)/include",
+        "<!(./bin/variables.sh cache_dir)/src/leveldb/include"
       ],
-      'ldflags': [
-        '-Wl,-rpath,<!(./platform/os.sh osdir)',
-        '<!(./platform/os.sh load_archive)'
-      ]
+      "sources": [
+        "./src/libbitcoind.cc",
+      ],
+      "conditions": [
+        [
+          "OS==\"mac\"", 
+          {
+            "xcode_settings": {
+              "GCC_ENABLE_CPP_EXCEPTIONS": "YES",
+              "GCC_ENABLE_CPP_RTTI": "YES",
+              "MACOSX_DEPLOYMENT_TARGET": "10.9"
+            }
+          }
+        ]
+      ],
+      "cflags_cc": [
+        "-fexceptions",
+        "-frtti",
+        "-fpermissive",
+      ],
+      "link_settings": {
+        "libraries": [
+          "<!(./bin/variables.sh bitcoind)",
+          "<!(./bin/variables.sh filesystem)",
+          "<!(./bin/variables.sh thread)",
+          "<!(./bin/variables.sh program_options)",
+          "<!(./bin/variables.sh system)",
+          "<!(./bin/variables.sh chrono)",
+          "<!(./bin/variables.sh libsecp256k1)",
+          "<!(./bin/variables.sh leveldb)",
+          "<!(./bin/variables.sh memenv)",
+          "<!(./bin/variables.sh bdb)",
+          "<!(./bin/variables.sh anl)",
+          "-lssl",
+          "-lcrypto"
+        ],
+        "ldflags": [
+          "<!(./bin/variables.sh load_archive)"
+        ]
+      }
     }
-  }]
+  ]
 }

cache/.gitignore

@@ -0,0 +1,3 @@
+patch_sha.txt
+src/*
+depends/*

etc/bitcoin.patch

@@ -139,14 +139,14 @@ index e7aa48d..df0f7ae 100644
  endef
 
 diff --git a/src/Makefile.am b/src/Makefile.am
-index 1c2f770..d2eca15 100644
+index 1c2f770..5582b7e 100644
 --- a/src/Makefile.am
 +++ b/src/Makefile.am
 @@ -1,6 +1,12 @@
  DIST_SUBDIRS = secp256k1
  AM_LDFLAGS = $(PTHREAD_CFLAGS) $(LIBTOOL_LDFLAGS)
 
-+lib_LTLIBRARIES =
++noinst_LTLIBRARIES =
 +libbitcoind_la_LIBADD =
 +libbitcoind_la_LDFLAGS = -no-undefined
 +STATIC_BOOST_LIBS =
@@ -179,7 +179,7 @@ index 1c2f770..d2eca15 100644
  TESTS =
 
 +if BUILD_BITCOIN_LIBS
-+lib_LTLIBRARIES += libbitcoinconsensus.la
++noinst_LTLIBRARIES += libbitcoinconsensus.la
 +LIBBITCOIN_CONSENSUS += libbitcoinconsensus.la
 +endif
 +
@@ -192,7 +192,7 @@ index 1c2f770..d2eca15 100644
    bin_PROGRAMS += bitcoin-cli bitcoin-tx
  endif
 +else
-+lib_LTLIBRARIES += libbitcoind.la
++noinst_LTLIBRARIES += libbitcoind.la
 +endif
 
  .PHONY: FORCE
@@ -203,7 +203,7 @@ index 1c2f770..d2eca15 100644
  	  $(abs_top_srcdir)
 -libbitcoin_util_a-clientversion.$(OBJEXT): obj/build.h
 
-+ARCH_PLATFORM = $(shell ../../platform/os.sh host)
++ARCH_PLATFORM = $(shell ../../bin/variables.sh host)
 +
 +libbitcoin_util_a-clientversion.$(OBJEXT): obj/build.h
 +clientversion.cpp: obj/build.h

package.json

@@ -2,7 +2,7 @@
   "name": "bitcore-node",
   "description": "Full node with extended capabilities using Bitcore and Bitcoin Core",
   "author": "BitPay <dev@bitpay.com>",
-  "version": "0.2.0",
+  "version": "0.2.0-dev",
   "main": "./index.js",
   "repository": "git://github.com/bitpay/bitcore-node.git",
   "homepage": "https://github.com/bitpay/bitcore-node.js",
@@ -27,8 +27,10 @@
     }
   ],
   "scripts": {
-    "preinstall": "./bin/build-libbitcoind",
-    "install": "./bin/build-bindings",
+    "install": "./bin/install",
+    "build": "./bin/build",
+    "clean": "./bin/clean",
+    "upload": "node bin/upload.js",
     "start": "node bin/start.js",
     "test": "NODE_ENV=test mocha --recursive",
     "coverage": "istanbul cover _mocha -- --recursive",
@@ -50,6 +52,7 @@
     "socket.io": "^1.3.6"
   },
   "devDependencies": {
+    "aws-sdk": "~2.0.0-rc.15",
     "benchmark": "1.0.0",
     "bitcoin": "^2.3.2",
     "bitcoind-rpc": "^0.3.0",

platform/os.sh

@@ -1,145 +0,0 @@
-#!/bin/bash
-
-exec 2> /dev/null
-
-root_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/.."
-BITCOIN_DIR="${root_dir}/libbitcoind"
-os=
-ext=so
-
-host=`uname -m`-`uname -a | awk '{print tolower($1)}'`
-depends_dir="${BITCOIN_DIR}"/depends
-h_and_a_dir="${depends_dir}"/"${host}"
-
-mac_response=
-check_mac_build_system () {
-  if [ "${ext}" == "dylib" ]; then
-    if [ ! -d "/usr/include" ]; then
-      if hash xcode-select 2>/dev/null; then
-        mac_response="Please run 'xcode-select --install' from the command line because it seems that you've got Xcode, but not the Xcode command line tools that are required for compiling this project from source..."
-      else
-        mac_response="please use the App Store to install Xcode and Xcode command line tools. After Xcode is installed, please run: 'xcode-select --install' from the command line"
-      fi
-    fi
-  fi
-}
-
-if test -f /etc/centos-release \
-  || grep -q 'CentOS' /etc/redhat-release \
-  || rpm -q --queryformat '%{VERSION}' centos-release > /dev/null; then
-  os=centos
-elif grep -q 'Fedora' /etc/system-release; then
-  os=fedora
-elif test -f /etc/redhat_release \
-  || test -f /etc/redhat-release; then
-  os=rhel
-elif uname -a | grep -q '^Darwin'; then
-  os=osx
-  ext=dylib
-elif test -f /etc/SuSE-release; then
-  os=suse
-elif test -f /etc/mandrake-release \
-  || test -f /etc/mandriva-release; then
-  os=mandriva
-elif grep -q 'Linux Mint' /etc/issue; then
-  os=mint
-elif grep -q 'Ubuntu' /etc/issue \
-  || grep -q 'Ubuntu' /etc/lsb-release \
-  || uname -v | grep -q 'Ubuntu'; then
-  os=ubuntu
-elif test -f /etc/debian_version \
-  || test -f /etc/debian-version; then
-  os=debian
-elif grep -q 'Arch Linux' /etc/issue \
-  || test -d /lib/systemd -a "$(readlink /usr/bin/vi)" = 'ex'; then
-  os=arch
-elif test "$(uname -s)" = 'SunOS'; then
-  os=solaris
-elif test "$(uname -s)" = 'AIX'; then
-  os=aix
-elif test -d /system && test -d /data/data; then
-  os=android
-fi
-
-os_dir=${root_dir}/platform/${os}
-
-if [ "${os}"  == "osx" ]; then
-  artifacts_dir="${os_dir}/lib"
-else
-  artifacts_dir="${os_dir}"
-fi
-
-thread="${artifacts_dir}"/lib/libboost_thread-mt.a
-filesystem="${artifacts_dir}"/lib/libboost_filesystem-mt.a
-
-if test -z "$os" -o x"$os" = x'android' -o x"$os" = x'aix'; then
-  if test "$os" = 'android' -o "$os" = 'aix'; then
-    echo 'Android or AIX detected!' >& 2
-  fi
-  echo 'OS not supported.' >& 2
-  exit 1
-fi
-
-if test x"$1" = x'osdir'; then
-  echo -n "$(pwd)/platform/${os}"
-  exit 0
-fi
-
-if test x"$1" = x'btcdir'; then
-  echo -n "${BITCOIN_DIR}"
-  exit 0
-fi
-
-if test -z "$1" -o x"$1" = x'ext'; then
-  echo -n "${ext}"
-fi
-
-if test -z "$1" -o x"$1" = x'thread'; then
-  echo -n "${thread}"
-fi
-
-if test -z "$1" -o x"$1" = x'filesystem'; then
-  echo -n "${filesystem}"
-fi
-
-if test -z "$1" -o x"$1" = x'depends_dir'; then
-  echo -n "${depends_dir}"
-fi
-
-if test -z "$1" -o x"$1" = x'h_and_a_dir'; then
-  echo -n "${h_and_a_dir}"
-fi
-
-if test -z "$1" -o x"$1" = x'host'; then
-  echo -n "${host}"
-fi
-
-if test -z "$1" -o x"$1" = x'load_archive'; then
-  if [ "${os}"  == "osx" ]; then
-    echo -n "-Wl,-all_load"
-  else
-    echo -n "-Wl,--whole-archive ${filesystem} ${thread} -Wl,--no-whole-archive"
-  fi
-fi
-
-if test -z "$1" -o x"$1" = x'artifacts_dir'; then
-  echo -n "${artifacts_dir}" 
-fi
-
-if test -z "$1" -o x"$1" = x'mac_dependencies'; then
-  check_mac_build_system
-  echo -n "${mac_response}"
-fi
-
-if test -z "$1" -o x"$1" = x'lib'; then
-  if test -e "${os_dir}/libbitcoind.${ext}" -o -e "${os_dir}/lib/libbitcoind.${ext}"; then
-    if test -e "${os_dir}/lib/libbitcoind.${ext}"; then
-      echo -n "$(pwd)/platform/${os}/lib/libbitcoind.${ext}"
-    else
-      echo -n "$(pwd)/platform/${os}/libbitcoind.${ext}"
-    fi
-  else
-    echo -n "${BITCOIN_DIR}/src/.libs/libbitcoind.${ext}"
-  fi
-  exit 0
-fi