Problem:

Since all the objects in AWS bucket are secured by default, but we need to access it through the URL to do certain things such as view images or read docs. The normal method is to use public URL to make it. But it’s not secure enough since everyone has unlimited access to it and easy to steal the information on the image.

List of solutions

• Go through all traffic from server-end

This is the normal way to get access to the object on the buckets. The client sends requests to the server with credential and server communicate with the bucket. After authentication, the server will send back the image requested by the client. This is a very direct way to get object but it’s inefficient because the image size is large and will eat up all the bandwidth. This method will quickly reach to the server traffic boot-neck and generate necessary load on the application server.

• Normal Public URL

This method is much faster and efficient than the first method because the client retrieves the full object file from S3 bucket without going through the application server. It is efficient but it is not secure since everyone can have unlimited access to the object as long as they have the URL, such as people can embed your file URL on their website.

• Presigned URL

A presigned URL gives the user/customer a permission to see their objects on the bucket without any security credentials or permissions. When the creator of the presigned URL create a presigned URL, the creator must provide their security credentials, a bucket name, an object key, client region, HTTP method and an expiration date and time. The presigned URL will include all those information on a single presigned URL. After the creator generated the presigned URL, user/customer can use this URL to see the specific object in a time period specified by the presigned URL. This feature is quite like the Snapchat burn after reading which gives users complete control of their data accessibility.

Challenges

During my work on generating the presigned URL. I think the biggest challenge is to generate random and unique s3key. Because of each object on the bucket should have different presigned URL, and the only way to differentiate them is the s3key. I use the UUID method to generate different s3key for each object on the bucket. After that, each object in the bucket will have a unique s3Key and never get overwrote.

Code snippet to do the Presigned URL

1: Add maven dependency

<dependency>
  <groupId>com.amazonaws</groupId>
  <artifactId>aws-java-sdk</artifactId>
  <version>1.11.360</version>
</dependency>

2: Create S3 Client

public class ServiceConfig {
  @Value("${aws.s3.bucket}")
  private String bucket;

  @Bean
  public StorageService getStorageService() throws IOException {
    AmazonS3 s3Client = AmazonS3ClientBuilder.standard().withCredentials(new DefaultAWSCredentialsProviderChain()).build();
    StorageService beanStroageService = new StorageService(s3Client);
    beanStroageService.setBucket(bucket);
    return beanStroageService;
  }
}

3: Generate the UUID for s3key

UUID uuid = UUID.randomUUID();
String randomUUIDString = uuid.toString();
String fileName = FilenameUtils.getBaseName(multiPartFile.getOriginalFilename())+"-"+randomUUIDString+"."+FilenameUtils.getExtension(multiPartFile.getOriginalFilename());

4: Generate the presigned URL

public URL getPreSignedUrl(String s3Key){
  java.util.Date expiration = new java.util.Date();
  long expTimeMillis = expiration.getTime();
  expTimeMillis += 1000 * 60;
  expiration.setTime(expTimeMillis);
  System.out.println("Generating pre-signed URL.");
  GeneratePresignedUrlRequest generatePresignedUrlRequest = new GeneratePresignedUrlRequest(bucket, s3Key)
    .withMethod(HttpMethod.GET)
    .withExpiration(expiration);
  URL url = s3.generatePresignedUrl(generatePresignedUrlRequest);
  return url;
}

5: Unit Test with Mockito

@Test
public void getPreSignedUrlTest(){
    storageService.getPreSignedUrl("IMG_0383-02208211-a987-4f3c-9822-a3ad6bc3b99a.JPG");
    Mockito.verify(client, times(1)).generatePresignedUrl(any());
}

6: Presigend Url example

"https://rent-service-dev.s3.us-east-2.amazonaws.com/
IMG_1372%203-33c73300-f077-4639-b999-f6f7e395d026.JPG?X-Amz-Algorithm=
AWS4-HMAC-SHA256&X-Amz-Date=20181023T160555Z&X-Amz-SignedHeaders=
host&X-Amz-Expires=59&X-Amz-Credential=AKIAIN367SIBE2ADRTQQ%2F20181023%2F
us-east-2%2Fs3%2Faws4_request&X-Amz-Signature=5c214058634b5c316e95665d8eb
b09662f1cef2c60bc38152307805d72dc7d09"

Quick Demo

I am a Java backend software developer. I have a strong background in software development including Java, Spring framework, micro-service, and AWS. During my working experience, I have gained lots of knowledge of Maven, Hibernate, Docker, Relational Database, PostgreSQL, GIT, Restful API, etc. I am always eager to dig deep into new knowledge and willing to improve myself every day.

GitHub Profile: https://github.com/dj458/RentService