以编程方式将文件从Azure Blob存储传输到Google云端存储(Transfer file from Azure Blob Storage to Google Cloud Storage programmatically)

 我有许多文件通过Azure数据工厂传输到Azure Blob存储中。 不幸的是，这个工具似乎没有为任何值设置Content-MD5值，所以当我从Blob存储API中提取该值时，它是空的。 
 
 我打算将这些文件从Azure Blob存储转移到Google存储中。 我在Google的Storagetransfer服务上看到的文档位于https://cloud.google.com/storage/transfer/reference/rest/v1/TransferSpec#HttpData，表示如果我提供的列表，我可以轻松启动此类转移。文件及其URL，长度（以字节为单位）以及每个文件的MD5散列值。 
 
 好吧，我可以轻松地从Azure存储中抽取前两项，但第三项似乎不会自动被Azure存储填充，我也无法找到任何方法来实现这一点。 
 
 不幸的是，其他选项看起来有限 到目前为止的可能性： 
 
 
 
 将文件下载到本地计算机，确定哈希值并更新Blob MD5值 
 
 
 查看我是否无法在可以计算散列值的相同区域中编写Azure函数应用程序，并将它写入容器中每个容器的blob 
 
 
 使用来自Data Factory的Amazon S3出口，然后使用Google的支持从S3导入，从https://cloud.google.com/storage/transfer/reference/rest/v1/TransferSpec#AwsS3Data获取，但这真的似乎浪费了带宽（我必须建立一个亚马逊账户）。 
 
 
 理想情况下，我希望能够编写一个脚本，打开并保持独立。 我没有Azure最快的下载速度，所以＃1会比想要的要少，因为它需要很长时间。 
 
 有其他方法吗？ 

I have a number of files that I transferred into Azure Blob Storage via the Azure Data Factory. Unfortunately, this tool doesn't appear to set the Content-MD5 value for any of the values, so when I pull that value from the Blob Storage API, it's empty.
 
I'm aiming to transfer these files out of Azure Blob Storage and into Google Storage. The documentation I'm seeing for Google's Storagetransfer service at https://cloud.google.com/storage/transfer/reference/rest/v1/TransferSpec#HttpData indicates that I can easily initiate such a transfer if I supply a list of the files with their URL, length in bytes and an MD5 hash of each.
 
Well, I can easily pull the first two from Azure Storage, but the third doesn't appear to automatically get populated by Azure Storage, nor can I find any way to get it to do so.
 
Unfortunately, my other options look limited. In the possibilities so far:
 
 
 
Download file to local machine, determine the hash and update the Blob MD5 value 
 
 
See if I can't write an Azure Functions app in the same region that can calculate the hash value and write it to the blob for each in the container 
 
 
Use an Amazon S3 egress from Data Factory and then use Google's support for importing from S3 to pull it from there, per https://cloud.google.com/storage/transfer/reference/rest/v1/TransferSpec#AwsS3Data but this really seems like a waste of bandwidth (and I'd have to set up an Amazon account).
 
 
Ideally, I want to be able to write a script, hit go and leave it alone. I don't have the fastest download rate from Azure, so #1 would be less than desireable as it'd take a long time.
 
Have any other approaches?

原文：https://stackoverflow.com/questions/41157327