Before I start talking about the uploader, I have to talk to you about how I joined the DTube team.
It goes back to 6 years ago! (Yes I know DTube to less than 1 year ^^).
It started with the meeting of @Heimindanger as an employee both in a company where have developed an eCommerce website.
We worked together a good year before he flies his own wings!
We kept a good distant relationship.
One day he contacted me and asked me if I wanted to participate in the project, he explains the project, I did not know Steem, IPFS, Meteor, crypto-currencies... I thought about it and I said yes!
The different versions
I started contributing to DTube on version 0.3 4 months ago with the possibility of translating DTube into several languages https://steemit.com/dtube/@heimindanger/dtube-0-3-this-update-contains-so-many-new-features-i-could-not-find-a-good-title-for-it.
Then, I hooked up with the creation of any part of the uploader for version 0.5 only 2 months ago https://steemit.com/dtube/@heimindanger/d-tube-0-5-big-optimizations.
Finally the possibility to have a preview of the video on the scrollbar, as well as the ability to send the Snap to resize and embed the new logo DTube, there is 1 month in v0.6 https://steemit.com/dtube/@heimindanger/d-tube-0-6-pushing-it-to-the-limit.
My first article!
If you followed, I'm quite new to Steem and it's also my first article on the internet, thanks in advance for your indulgence ^^
The subject of this article is about the uploader! I decided to do this article because many people are wondering about the operation, I thought it was a good reason to dive into the bath for this first article!
Why uploader?
To answer problems that users were facing and because it was not possible easily in decentralized mode!
The first need was to encode the source videos in a smaller format (480p) allowing more people to play videos with a low internet speed.
The 2nd was to have the possibility to have a progress bar when sending the video file.
We then took advantage (having on hand the source video!) to encode the video in a video and audio format allowing to be read by the largest number of browsers on the market (H264 for video and AAC for audio).
And this allowed us to take the opportunity to directly send the source video and 480p video directly into IPFS and thus lighten the customer of this long and unstable task at the time.
In version 0.5, we then added the creation of a video Sprite allowing to visualize thanks to the scroll bar of the video the preview of the moment of the video.
Let’s go!
Here is the block-by-block explanation of the operation of the uploader that allows you to send videos to DTube!
Upload video
b/ Sending the video to the server with progress bar
f/ Token recovery for encoding tracking, IPFS hash recovery, ...
The heart of the Uploader
For fast processing after the end of the upload, everything is treated as asynchronously and in parallel, there is only one token that is returned with prior requests for creation of Sprite (d), encoding (e) performed …
Encoding Daemon
Daemon encoding can handle 2 different tasks, creating Sprites files and 480p encoding of the source file.
How the Sprite is created to preview the video:
- Extraction of 100 images of the source video in 210 * 118 respecting the ratio
- Combining 100 images vertically => allows to see the first images if the image is slowly loading!
Note: if the video is less than 100 seconds, the extraction of images is done with 1 image per second, which is more than enough.
For 480p encoding operation, the source video is simply converted to 854 * 480 for 16: 9 video.
Video conversion to H264 is also performed as well as audio conversion to AAC. Thus this 480p video can be read by many people and by several browsers, unlike the source video.
Once each file is ready (Sprite or 480p file), it is placed in the IPFS queue.
The progress of each treatment is updated every second so that the user can follow it.
IPFS Daemon
The file is added to the decentralized IPFS network.
Once the user recovers all the hashes via repeated calls, the video article can be posted on the Steem network.
Finally on the home page
The video list is extracted from Steem's blockchain, and each video contains an url link from IPFS, hosting the Snap, video, Sprite...
Global scheme !
Extra: the Snap and the DTube logo inlay
Steps:
- Simultaneous scaling and cropping for a final image of 210 * 118
- Adding in IPFS this final image
- Adding IPFS to this image with the DTube logo overlay
Optimizations
If a user had to leave the upload page after sending the video, 480p video encoding and adding to IPFS would be useless and valuable resources would be lost.
That's why if the upload page is closed, the tasks associated with this source video on the video encoding, the creation of the Sprite, the addition in IPFS ... are canceled!
This point saved a lot of resources several weeks ago.
Limitations
The 480p encoding and the creation of the Sprite is for the moment limited to videos less than 20 minutes for cost reasons. Which is still the majority of videos sent on DTube.
The encoding of the source video is currently limited to 480p for cost reasons as well.
Note: These are things that will evolve and change in the future!
The source video
The source is kept as is, without any transformation and limitation in format, size, duration ... but we must pay attention to browsers and Internet speeds of users who may not be able to view the video and not vote at once!
A video with a smaller size, and an encoding in h264 for video and AAC for audio will allow all to see your source video, think about it!
Shortly
The uploader is evolving!
Since the last version v0.6 things have accelerated, the affluence helping and the technical challenges that grow with, and it's rather motivating!
The next article will explain these changes.
Conclusion
This is my very first article that I publish on the internet and also on the blockchain Steem, be indulgent!
Thanks to @heimindanger for his tips for writing articles!
Feel free to leave your comments. I’m french, so thanks to google translate!
Links
DTube: https://d.tube
Uploader: https://github.com/dtube/ipfs-uploader
I know how hard you guys go through with all these. I’ve been using Dtube since I learned about it there still a lot to improve here but i’ll be glad to see dtube be in a different level soon. Thanks for all your hard work to make dtube progress and ensure quality.
Yes, thank you
Why can I not search d.tube or post vids from my iPhone?
Yey!
Dtube is growing on the inside let's feed it with the original content
Thanks for sharing the inside on the improving of the Dtube's uploader. Tried it out today and had a very positive smooth experience. I believe that as much as Dtube needs to constantly upgrade its tech it should also be looking towards raising the quality standards of its platform, potentially looking into creating its own original content.
And I am extremely happy to start contributing content to my Dtube channel. I'm currently working on a documentary about the Crypto Utopia being built in Puerto Rico currently and am wondering would you guys (@dtube, @heimingdange, @superkoala and others) would like to discuss the potential of launching it as a Dtube original project.
This episode is a pilot for the web-documentary about the way Crypto changes our world. Shot in the lean format this project is able to catch the trends faster than the other traditional big media sources documenting the history in making.
Merci,
-K.
It's pretty amazing to see the behind the scenes of the video processing that DTube and the uploader do.
People don't realize how complicated this stuff really is, especially when dealing Peer To Peer. Thanks for all your hard work and I can't wait until we see even more improvements and quality adjustments!
Yes and it's a team work!
Welcome , you did pretty good for your first article and also thank you. If dtube is going to take over its going to need all the brain power it can get, much success.
Thank you
Just a thought. Maybe a Windows and Linux self installer and auto updater for those of us who wish to contribute processing power? Say I want to donate 20% or so of one of my processing cores and 10% of my GPU to process videos.
Additionally an option for redundancy for video files itself, not so much like Torrents but very similar like how Freenet Project handles websites/FreeSites for sake of example?
I'd willingly donate some processing power from my gaming rig and roughly 50 gigs of HDD.
Another approach is in progress to contribute.. a desktop version in windows linux and ios to make what the uploader do but on your computer!
You can also run a ipfs node to contribute for the ipfs network if you want
I'm aware I could contribute to the IPFS network voluntarily, what I'm getting at though is that there are likely people who would contribute in some size, shape or form, but when it comes to installing and setting up IPFS they get deer in headlights syndrome. Even some of the more technically savvy end up that way ;)
So all the same, having an installer that would essentially do the basic installation would be beneficial. Or does one exist now since the last time I checked and I'm just being an idiot? =\
Is there a tutorial on how to do this? Also if you host your own stuff on your own IPFS node can you achieve greater than 480p? Thanks for your awesome work!
Yes there is an tutorial, i found one for german:
Huh https://dist.ipfs.io/#go-ipfs
Wouldn't you know it =\ things have progressed a hell of a lot since the last time I peeked at this stuff.
You really landed with the right foot into the Blockchain. Great improvements DTube is having... as always, everything is a matter of time!
Thank you!
Thank you for your AMAZING work on Dtube it's a HUGE part of this community!
Welcome to steem @superkoala! :)
And thank you for your contribution to dtube!
Thank you!
@superkoala Man, you guys are amazing!!! I have seen the progress from the beginning, and it is GREAT. Looking forward to big things, and I am recruiting to DTube every chance I get. :-)
If I may ask I'm very curious as to the availability of infrastructure that DTube has? As you may know, YouTube has 400 hours of video uploaded every minute. Granted, I don't anticipate you guys dealing with that load factor anytime soon, but I'm curious how you guys are set up.
I'm incredibly excited about the future of DTube, however I'm keeping a cautious outlook as I've seen platforms try and compete with YouTube, although I'm not suggesting that's what you guys are trying to do, I've seen platforms have to close shop because they just didn't have the capacity to handle all the traffic.
Thanks for providing this environment for us. My experience here has been phenomenal and I'm very excited about contributing and participating.
Thank you! You are right and we know that very very well 😀 it is our first priority since the start and we work hard on it. If you see now it is ok. We increase hardware when we reach a limit for cost reason and we have a lot of idea to manage more and more trafic. We have idea to increase trafic too but we delay it to keep the platform working 😉
When will dtube support videos on 720 or 1080 ?
@superkoala why your post is always resteem by @Dtube i am also a @Dtube user he never resteem my post ever ?
@sharolyn it was the only one time when my first post was resteem by dtube, why you said always ? What do you mean ? Also my post explain how work the uploader and I am the developper of the uploader, and the uploader is a part of dtube
@superkoala i mean that what kind materiel @Dtube want to resteem i am new one here thats why i ask if you'll guide me it'll be your kind. it's really hard for me get upvoter even my materiel is not bad
Yes your post are not bad ! Dtube never resteem post, except those made by heimindanger and mine to explain what we do in Dtube because we are developper of Dtube
I believe that Dtube is next station for all vloggers Because of it's innovation and being easy to use :)
We were waiting this solution long time ago to be the best alternative for adsense as adsense is not transperant.
Great work Guys :D
Its very good ! and interesting :)))
The platform is nice. I like D.tube more then youtube.But it's the same "be popular" think like in youtube. If i dont have friends i can't get even dollar:) But i will tray how best i can.
I know this is slightly off topic but still related to Dtube. Does anyone else have a real difficulty trying to get videos to play smoothly? Every time I try to watch a video it either freezes after a few seconds or starts doing a really jumpy buffering where audio comes in and out and the video plays for like 1 second and then stops for a second and this just goes on and on. I found an app on PlayStore today for Dtube so I thought I would try that to see if this was any better but although the issues were slightly different it was still pretty much unwatchable. It's a shame as this app had support from Chromecast too. Before anyone asks about bandwidth I have 80Mb down and 50Mb Up so this isn't the issue and YouTube vids play just fine. Hope this is something that can be fixed as I really wanna watch some of these videos.
It is very strange with your bandwith, where do you come from ?
Dtube is great and i would really love to use it more but some competitions ive entered discourage using it due to issues. Even on some smartphones it seems videos just will not start.
Youtube is starting to be universaly hated across the board, from creators to users, and dtube really has an opportunity here and now.
I really hope you seize it. 😁
I hope too, we work hard on it!
please upvote
https://steemit.com/dtube/@syukranpta/4sy9d5c9
Great job! d.tube is looking and working great. I hope d.tube is successful, maybe you can give youtube a run for it's money ;)
useful postings. I like your post .. thanks
Really cool job!
Nice to show that big part of your life to us
It would be interesting how the encoding process works so some can perform it on there machines and upload the product directly to Ipfs. I mean you could also put it on YouTube and download it afterwards but I don't want to feed YouTube with my videos.
Ok after thinking of my answer I could just read the documentation of ffmpeg and work on a script.
Hello, I'm a dtube beginner. I hope to learn a lot from you about dtube.
My next hope is to buy the latest laptop, because my laptop is now broken, I use it for 6 years, the output of 2012
T_T
this information is very vital we need to let it go viral by resteem it's perfect
Rather informative and I was honstly curious how that actually worked. Thanks for the blow by blow on the uploader and how videos were done. Informational itch has been scratched.
Yes I try to keep it easy to read and to understand because it is a little bit complicated^^
Far more complicated than the stuff I've worked on in the past, that much I'll say. Damn dude I'm a physicist and this kind of process warps my brain a little bit trying to think deeper into the nitty gritty. I commend you on your hard work, dedication and devotion to the projects.
Thank you!
Good posting
Wow, thanks for the information! Always you give good information that's why i follow you!🔝🔝🔝🔝
Thank you!
Great job and welcome to steemit! :)
Maybe you can help me out, I've never been able to upload any videos edited using Windows Movie Maker. Is this because WMV files are not supported?
Thank you! You can report this here : https://discordapp.com/channels/347020217966395393/400281011826458635
Using C# for the encoding daemon I see … "Interesting" choice.
I'm wondering how many servers you're running right now for the encoding service (and where. Azure? AWS?). How well does it scale, etc.
Yes C# forever :-) Now, we have 3 to 4 server for encoding with a third party which accept steem ^^
I have a question about the local IPFS, the one an user can activate in order to host other user's videos to alleviate some of the gateway's traffic on DTube's end. Do you know how we can install it?
I have checked some tutorials, but the link they provided to install it is broken :)
Thank you so much, I'll check it out once I go back home!
Can't wiat for that. I would probably offer some space/bandwith to seed videos (about 1TB, 10MB/s upload). Right now I use webTorrent but to be honest it works a bit weird :|
If I join @Dtube or upload videos to @Dtube will I still retain full ownership of the video? Will I be able to delete the video if I choose to?
I think you can't delete it, but I'm not the best person to respond to that
Post yang bagus...
Tapi saya tidak pandai untuk membacanya...
Good decision to work together with him.
thanks for your information
Thank you for your work on Dtube! Great article... for an internet publishing newbie... Ha ha!
Yes it is really my first article. But @heimindanger help me with tips and advices, thanks again for him
Very well explained! And great graphics.
Thank you! Powerpoint power ahah!
Glad to have you on the team! I'm really looking forward to being able to use DTube, because I've been unsuccessful the past fifteen times I've tried to upload... Error submitting to blockchain. I am able to upload video, then upload the snap, but when I enter the text description, etc, and click submit, i get the blockchain error... Until then I will continue using Youtube, since I do 4K UHD drone videos. Cheers.
I’ve written about this issue and how to get through to upload your video to @dtube
Check this out @texagonia
https://steemit.com/dtube/@blascobite/dtube-upload-error-and-how-to-get-through
hummm... I think it is not uploader issue ... but steem, try report here : https://discordapp.com/channels/347020217966395393/400281011826458635
thx for this info...
Nice work
This is amazing! your post is the best many Dtubers can learn about this like me. Thank you for the contribution Dtube
Thank you
Hello, I need some help with d.tube. Sor far, I can use d.tube for watching rating videos and comments. But when it comes to write or replay to a comment below a video on d.tube, it doesn't work for me. When I click "add a comment" or "reply to..." it doesn't do anything.
I use Chrome with a script blocker so I thought first maybe it blocks something, but still, after I switched it off, it didn't work. Another person recommended me to use steemconnect instead of my private posting key but still no success. So far I've been writing comments directly on steemit.com but I would prefer to use the d.tube interface. Do you have some other ideas or tips for me?
It is a front web side issue, perhaps @heimindanger or @hightouch can help you
I found the source for the problem. After deactiviating the extension "Grammarly for Chrome" it works now. Obviously d.tube uses a different text area than steemit.com.
Thank you for sharing
would you be able to help me too please? I am trying to upload a video to dtube and it says i must upload a snap , i do that and it just keeps going round and round... any help please!!
very good post you i will try the way that post
I'am new be here, i want lear abouth dtube but i don't know how i can learn abouth this
You can read the about page of dtube!
Awesome. So cool to see how it's running. Lots of work goes into it and we appreciate it!
I got some setting from your article that's I'm just going to convert to before I upload.
Thank you for the details of this amazing work. I just got started with posting my videos and I would like to understand what video format you recommend uploading at (the original video), because I've tried avx since it has a much smaller footprint but then I lack the ability to take a snapshot since the upload window doesn't display the video itself. Maybe avx is not a good idea. Would love some pointers and thanks again for the fine work.
Like I write in this article, we recommend h264 video encoding and aac audio encoding
good, vollow & votes @wannis
Can anyone please help explain how to use dtube. Im new
See dtube about page ☺
DTube problem. on my iphone I can not click the search button on d.tube. the icon is there, but there is no function.