Task:
Perform the AWC CLI client installation, set-up/configuration, and perform synching.
A. Installation:
1a. For Windows, download the client.
e.g. awscli.amazonaws.com/AWSCLIV2.msi
Afterwards confirm installation with:
C:\> aws --version
<read output and version installed>
1b. Linux Package version:
Note:
- Ubuntu and Rocky OS/Centos Stream/Red Hat Linux have packaged maintained version. If not having the most current release is not required, simply install it using the system package manager.
e.g.
Ubuntu:
- $ sudo apt-get install python3 python3-pip unzip awscli
Rocky OS / CentOS Stream:
- $ sudo yum install awscli
Afterwards, verify the version:
$ aws --version
<confirm version>
1c. Linux Latest version:
Note:
- If you require the current/latest release, you have to uninstall the package maintained version, and perform an alternate install via download or Snap.
- Obtain the current configuration:
$ which aws
<read output - e.g. /usr/local/bin/aws>
$ ls -l /usr/local/bin/aws
<read output - e.g. /usr/local/bin/aws -> /usr/local/aws-cli/v2/current/bin/aws>
Note the two locations for the install step below.
- Remove the CLI package version, if installed:
Ubuntu: $ sudo apt-get remove awscli
Rocky OS/CentOS Stream: $ sudo yum remove awscli
- Download, unzip, and install the newest version via Download:
e.g. As of 2026/04, location is: awscli.amazonaws.com/awscli-exe-linux-x86_64.zip
$ cd /home/myadminid/
$ curl "<https-download-url>" -o "awscliv2.zip"
<wait briefly>
$ unzip awscliv2.zip
<watch extract>
If this is the first install of the aws client, simply run:
$ sudo ./aws/install
or
If this is an upgrade, we'll want to re-use the old directories. So we need to include them.
e.g.
$ sudo ./aws/install --bin-dir /usr/local/bin --install-dir /usr/local/aws-cli --update
Remove the zip and installation files, if desired.
$ rm awscliv2.zip
<confirm>
$ rm -rf aws/*
<verify and confirm>
B. Configuration:
The configuration depends on your needs, the simplest methods are the Short-term credentials and the re-using the EC2 instance config.
For multi instance management, use roles set-up in the IAM Identity Center (SSO), then configure your instances using an IAM role. See the CLI user guide for more information.
Option 1: Re-use your current Management Console login:
For small set's of instances, simply use your current AWS Management Console login.
Notes:
- Minimum CLI version for the CLI aws login command is 2.32.0
- If the EC2 instance does not have a web browser (e.g. server w/o a GUI), use the --remote option and adjust the arn:aws:signin as below. If you don't add the --remote option, you can copy-and-paste the URL (signin.us-east-1.amazonaws.com/authorize?blabla) to your workstation browser, as well.
- This will create a profile, and save an authorization code for short-term use. This is not ideal for scheduled syncs, use the next Option 2.
If your EC2 instance has no web browser, enter:
$ aws login --remote
If no profile is found, the CLI will prompt for your region.
...
You are able to change the region in the CLI at any time with the command
`aws configure set region NEW_REGION`.
AWS Region [us-east-1]: us-east-1
The CLI will wait for the authorization code to be pasted/typed into the CLI:
Browser will not be automatically opened.
Please visit the following URL: ... signin.us-east-1.amazonaws.com/authorize?blabla
Please enter the authorization code displayed in the browser: <enter the code here>
Copy the URL to your workstation browser, and authenticate using your normal Management Console login. After login, copy the authorization code back to the EC2 instance.
If you current login overrides another, the CLI will prompt you to overwrite it and use the new profile instead. Answer (y/n): y
Option 2: Create an authentication key in the Management Console and configure CLI to use this profile.
1. AWS root/admin login --> IAM --> Users
2. Create new user
- Name and complete user. Note the new user's arn ID.
- - Click the copy icon next to the arn to copy to clipboard. Paste the arn into your secure id and password repo/program for later use below.
3. Add Permission Policies:
<current user account page> --> Permissions (tab)
- Add roles. e.g.
- - AmazonS3FullAccess
- - AmazonS3OutputsReadOnlyAccess
- - AmazonS3ReadOnlyAccess
- - . . .
4. Create the Access key:
<current user account page> --> Security credentials (tab) --> Create access key (button)
- Copy the Access key ID and the Secret access key.
WARNING: You will NOT be able to access the key again. You'll have to delete and re-create the key if you lose it.
5. Configure the AWS CLI client on the workstation or server using the key:
$ aws configure
<answer the prompt below with the info gathered above>
AWS Access Key ID: <enter the key ID here>
AWS Secret Access Key: <enter the key's key here>
Default region name: [us-east-1]: us-east-1
Default output format: [json]: json
Verify the new profile created:
$ aws sts get-caller-identity
<review json and confirm the userId, Account, and Arn account>
Verify you can see your S3 bucket:
$ aws s3 ls
<review storage bucket list>
C. Manual Sync from Local Server to S3 cloud:
Option 1: Manual Sync using current credential (not the access key):
$ aws login --remote
<follow prompts>
$ aws s3 sync /local/folder/path/ s3://my-bucket/folder/path
<wait/watch output and confirm synced files listed>
Option 2: Manual Sync using the access key/profile created above:
$ aws s3 sync /local/folder/path/ s3://my-bucket/folder/path
<wait/watch output and confirm synced files listed>
WARNINGS:
- S3 sync is NOT bidirectional. The updates only go one way, either up or down. If changes are being made on both ends, conflicts are likely. Keep a simple directional workflow or a naming convention if possible to avoid conflicting files.
- If files stay the same size, and are often updated, you may not get all updates. To ensure all modifications are transferred, append the --exact-timestamps flag to the aws s3 sync command. This will allow precision down to milliseconds.
Notes:
- The sync is like rsync, it syncs files that have changed (been modified locally since last sync) or are new files. Syncs after the first sync are quicker because only new/updated content is uploaded to S3.
- It may take a few seconds or a minute for the files/updated files to display in the S3 bucket in the Management Console.
- Use --exclude and --include to restrict what is synced from the source to the target
- Use --delete to tell the CLI to remove files on the target S3 bucket not in the current source folder
Option 3: Manual sync the first time from S3 to the local workstation/server:
$ aws s3 sync s3://my-bucket/folder/path /local/folder/path/
<wait/watch output and confirm synced files listed>
Note:
- If the local/folder/path/ doesn't exist, the CLI will auto-create the folder(s).
D. Scheduled Sync from Local Server to S3 cloud:
Notes:
- Create a .bat file (MS Windows) or .sh file (Linux) that performs the sync.
- In MS Windows, use the Task Scheduler program.
- In Linux, we have:
- - User-specific jobs, scheduled via crontab -e
- - All the cron folders: /etc/cron.d, /etc/cron.daily /etc/cron.hourly, /etc/cron.monthly, and /etc/cron.weekly
Option 1: If running as a current user via crontab -e:
- Create the file:
$ vi ~/awsclischeduledsync.sh
aws s3 sync /local/folder/path/ s3://my-bucket/folder/path --exact-timestamps
...
<esc>:wq (to save)
- Add the schedule/file at the bottom of your current list of cron jobs:
$ crontab -e
<this will open an editor to define/update your schedules>
. . .
0 0 * * * /home/myadminid/awsclischeduledsync.sh
<esc>:wq (to save)
Option 2: If using a system cron folder:
Note:
- Create the sync execution file in a place where the system will have at least read and execute access.
- Add to the desired cron folder by creating a cron entry in the folder and specify it to run the execution file.
Example folder where admin/user with access to the system sbin folder can kick-off:
$ sudo vi /usr/sbin/awssync-cron.sh
#!/bin/bash
# perform sync
aws s3 sync /local/folder/path/ s3://my-bucket/folder/path --exact-timestamps
<esc>:wq (to save)
Example to run daily at midnight:
$ sudo vi /etc/cron.daily/awssync.cron
0 0 * * * /usr/sbin/awssync-cron.sh
<esc>:wq (to save)
previous page
|