Repository navigation
Concerns with backends #30
Description
I can't get the logic of cache & store backends. In refile file is placed into cache backend until record is stored, but it's working slightly different here.
I'm implementing shot processor now, which is about taking frames of the videofiles due to specific time of video. I haven't extracted its logic from model yet.
The overall code looks like that (I'm inserting process_shot method to changeset):
defp process_shot(changeset, file) do
thumbnail_time = Ecto.Changeset.get_change(changeset, :thumbnail_time, "0")
video_path = Exfile.Phoenix.Helpers.exfile_url(MyReelty.Endpoint, file) # the problem here
temp_path = "/tmp/#{Ecto.UUID.generate}"
System.cmd("ffmpeg", ["-i", video_path, "-ss", thumbnail_time, "-vframes", "1", "-f", "image2", temp_path])
shot = %Exfile.LocalFile{
path: temp_path
}
cast(changeset, %{ shot: shot }, [:shot])
end
Here is the problem. I have a exfile-b2 installed, and there are configs:
backends: %{
"store" => {ExfileB2.Backend,
hasher: Exfile.Hasher.Random,
account_id: ":P",
application_key: ":P",
bucket: ":P"
},
"cache" => {Exfile.Backend.FileSystem,
directory: Path.expand("tmp/attachments")
}
}In the changeset I'm supposing file to be stored in the local system according to cache backend system, but it doesn't. That's why my code makes roundtrip by downloading video from b2 and then uploading its shot back. This is definitely big price in the terms of both time and price for the traffic. What's wrong with cache backend? How can I deal with my situation?
Activity
keichan34 commentedon Jul 20, 2016
Right now, when you upload a file via the multipart method, the file will be directly uploaded to the
storebackend. To upload the file to thecachebackend, you have to use thePUT /attachments/cacheendpoint to upload the file outside the context of an ecto transaction.In your case, you may be able to upload the file using that endpoint, set the file ID in to the changeset, then when you're done processing, you can upload it to the store backend. It may be a bit easier to write an
Exfile.Processorand put it in the list of preprocessor, though -- those will run before the file is uploaded.I agree, this isn't ideal -- I made a mistake by uploading the file immediately when the file is assigned to the struct using
Module.change/3. I'll deal with this in #31 by automatically uploading to the cache backend first, then uploading to the store backend after the save to the DB has succeeded.