Skip to content

Hosting notes#

The plugin behaves the same everywhere; servers do not. These notes come from the first deployments of the plugin against real staging servers and say what to check before you rely on it on a new host.

File ownership and the PHP user#

A file push is written by PHP, as whatever user the web server runs PHP as. Your SFTP or SSH account is usually a different user. Two things follow:

  1. PHP can only write where its user has write permission. A theme folder that was uploaded over SFTP by another account is often read-only to PHP.
  2. What PHP creates belongs to PHP's user, and must stay editable by your SFTP account.

Symptom. A push stops on the first file with

The remote could not write N of M files (first: …). Its folder for this scope is not writable by PHP there (ownership / permissions), or the path is refused.

or, for a large file sent in chunks, Write refused for <path>: …. Versions before 0.2.1 reported success with nothing written.

What the plugin does. A push leaves files at mode 664 and new folders at 775, so a second account in the same group can edit what a push wrote.

What you have to do once per server. Put both accounts in one group and make the scope folders group-writable.

Example: Bitnami#

On a Bitnami WordPress stack PHP runs as daemon and the login account is bitnami. A theme uploaded by bitnami is bitnami:bitnami with files at 644: PHP cannot write to it.

# once: let the login account share the web user's group (takes effect at the next login)
sudo usermod -aG daemon bitnami

# for each folder BB Migrate should write: the theme, a plugin
sudo chown -R bitnami:daemon wp-content/themes/my-theme
sudo find wp-content/themes/my-theme -type d -exec chmod 775 {} +
sudo find wp-content/themes/my-theme -type f -exec chmod 664 {} +

A file uploaded over SFTP afterwards is again bitnami:bitnami 644 until it is fixed, and the next push that touches it reports "Write refused". Pick one way to deploy a folder and keep to it.

Other hosts#

  • Find the PHP user: ps aux | grep -E 'php-fpm|apache|httpd', or compare the owner of a file WordPress itself wrote (a recent upload) with the owner of your theme folder.
  • Some managed hosts have a control panel action that resets file ownership and permissions for an application (Cloudways calls it "Reset File Permissions"). Run it before the first push.
  • Before using BB Migrate on a production server, compare the owner of the theme folder with the PHP user. If they differ, fix ownership first.
  • The uploads folder is normally writable by PHP already, since WordPress writes media there. The storage folder and the replay markers live under it. If uploads/bb-migrate/nonces/ cannot be written, every request is refused as Replayed request.

Request size limits at proxies#

Every request is a JSON body. Something between the two sites (a CDN, a WAF, a load balancer, the web server's own body limit, post_max_size) may reject or truncate a body it considers too large.

Symptoms.

  • Chunk arrived without a path or data: the request body was probably truncated by a proxy or firewall (N bytes received). Lower "Max request size" on the sending site.
  • Request failed: cURL error … with a TLS read error part-way through an upload.
  • Unexpected response (413) … or an HTML error page from the proxy.

How max_request relates. max_request is the plugin's own ceiling for one request body, 1 MB by default.

Transfer Whose max_request Effect
File push The sender's Small files are batched up to it; a large file is cut into chunks of half of it (never below 64 KB)
File pull Both The sender's sizes the requests; the remote's caps each response of small files
Database push The remote's, as reported in the handshake The number of rows per chunk is halved when a chunk exceeds it and doubled when a chunk is under a quarter of it
Database pull Neither Fixed thresholds (about 700 KB and 200 KB per chunk) steer the number of rows per chunk

So for a file push through a strict proxy, lower Max request size on the sending site (Settings tab). In one deployment behind a CDN, 131072 (128 KB) was the value that worked for ordinary files. For a database push, lower it on the receiving site.

max_request is a target, not a hard cap: a database chunk never holds fewer than 25 rows, however wide they are, and the base64 estimate for files is approximate.

Large media. In the same deployment, chunked uploads of files of a few megabytes failed with TLS read errors through the CDN regardless of the chunk size. Since then the client retries idempotent file requests up to three times (Protocol). If a file still does not get through, send it straight to the origin (below), copy it by another route, or check whether it is already there and identical.

The 100 second response cap behind Cloudflare#

Cloudflare gives the origin 100 seconds to answer a request. After that the visitor, here the sending site, gets an error page, whatever the origin is still doing. Other proxies have similar caps with other numbers.

Most BB Migrate requests are short by construction: a chunk of rows, a batch of files, a 12 second backup step. Three are not, because they do their work in a single request:

Request When it is long
content-import A package of many posts, or many attachments whose image sizes are generated
import-finalize Find and replace across a large database after the swap
site-snippets Many snippets

The sender waits up to 600 seconds for the first two, but the proxy cuts the connection at its own limit. The receiver keeps working after the caller is gone (requests run with ignore_user_abort), or is stopped by its own PHP limits; either way the sender sees an error and has no report. These requests are never retried.

What to do.

  • Send page packages in small batches. A few dozen posts per push. In the first deployment a single import of about a hundred posts did not complete; batches of at most fifty did.
  • Or go straight to the origin. The signature covers the route and the body, not the host name, so any URL that reaches the same WordPress works in a connection string. Use a host name that resolves to the origin and is not proxied. TLS verification still applies to that name.
  • Check afterwards. After an error on a long request, look at what arrived (wp bb-migrate pages list, the remote's admin) before sending anything again. Re-pushing pages with --on-exists=update is safe to repeat.
  • For a large database, prefer pulling from the destination side when you can: on a pull the long steps run locally under WP-CLI, with no proxy between the work and its result.

Opcache windows after an update#

PHP's opcache keeps compiled files in memory and, on many hosts, checks the file on disk only every so often (opcache.revalidate_freq; 60 seconds on the Bitnami stack mentioned above). After PHP files change on disk the server may run the old code until that window has passed. Some hosts disable the check entirely and need a PHP reload.

This matters in three places:

  • After pushing a theme or plugin with file sync: wait for the window before the next step that depends on the new code, such as activating the plugin or purging a cache that the new code fills.
  • After uploading a new BB Migrate zip: for up to the window, the receiver may still answer with the old version. The sender logs Remote runs BB Migrate X, this site Y until the new code is live.
  • The compatibility mu-plugin is a copy. It is refreshed when its content differs from the bundled file, on the next admin page load or WP-CLI run on that site. After updating the plugin on a remote, open its settings page there once.

On the command line, PHP has its own opcache settings. The plugin registers its WP-CLI command only after WordPress has loaded, which avoids a known fatal ("Cannot declare class WP_CLI") seen with opcache.enable_cli on and WP-CLI as a phar on some PHP builds.

Nginx serving uploads statically#

The storage folder wp-content/uploads/bb-migrate/ holds database dumps. The plugin protects it with a .htaccess that denies all access. That file is honoured by Apache. Where nginx (alone, or in front of Apache for static files) or a CDN serves the uploads folder directly, .htaccess is not read and a file in the folder is served to anyone who knows its URL.

This is why:

  • storage names are random. Every dump, backup and package has 32 random characters in its name, and the folder has an empty index.html so it cannot be listed;
  • storage names are withheld. No file name in the folder is ever reported to another site. Before 0.2.2 the name of the pre-push backup was returned to the pushing site, which on such a host gave a key with only Accept push the whole database.

What to do on such a host:

  • Add a server rule. For nginx:

    location ^~ /wp-content/uploads/bb-migrate/ {
    deny all;
}

  • Keep the retention short ("Keep exports/backups for (hours)" on the Settings tab), and delete dumps you no longer need from the Export tab.

  • Check from outside: request https://example.com/wp-content/uploads/bb-migrate/index.html. A 403 means the folder is protected; an empty 200 means files in it are being served.

Other things that differ between servers#

  • Clocks. A signature is valid for 300 seconds either way. A server whose clock is off by more than that rejects every request with Signature expired; check both clocks. Run NTP.
  • Permalinks. Without pretty permalinks the REST API is at ?rest_route=. Use the connection string the site itself prints.
  • Firewalls and security plugins that block or challenge unauthenticated POST requests to /wp-json/ produce Unexpected response (…): (an HTML page, not the plugin …). Allow the bb-migrate/v1 namespace.
  • Page caches must not cache REST responses. The routes are POST, which caches normally pass through.
  • Third-party plugin versions. Site actions call into BB Cache and Custom CSS & JS on the remote. A different version there can behave differently from the one you tested against: in one deployment an older version of the snippet plugin on the receiver did not evaluate a rule type, and a correctly pushed snippet did not print until that plugin was updated.