YouTube-esq module for Menalto Gallery v3

Advertisements

I’ve been using Gallery2 on my site now for a few years now. I’ve always had issues with it though, due to it’s somewhat overwhelming complexity, the templating system which I really couldn’t even get a grasp of, and based on some framework I really didn’t understand, I got reluctant to attempt to do anything with the code in the way of writing modules for it. That and file permissions issues plagued my gallery and stopped it working on several occasions.

So I read that Gallery3 was almost ready, and installed the RC2 version. It’s much better in the terms of the way it’s built. It’s modules are simpler to write with a somewhat simple ‘hooks’ based system which actually make extending it’s functionality and hooking into the theme’s areas relatively easily.

One of the main things that I’ve always wanted to do is have my videos stored on my Gallery instead of on YouTube. I get more web-visitor stats from the server, Gallery, and Google Analytics – something I don’t get with YouTube. However, G3 doesn’t support this out of the box. So I wrote one. It’s a somewhat basic implementation of what I eventually want it to be. The biggest part of this for me was trying to get to grips with Gallery3’s new framework (based on Kohana). I figured most of it out by copying the basic structures of other modules I’d installed and modifying it for me own needs. A couple of hours and I’d picked it up relatively easily.

You can download it here: [download id=15 format=1]

I probably shouldn’t be doing this, but in the interest of avoiding dead links, I’ve been updating this post during the development of this module as opposed to posting a new post each time there was a new version.

A couple of things you should note however. If you want to upload a video of more than a couple of megabytes, use the ‘Server Add’ module, as the upload will eventually stall/time out/crash as the transcoding process takes a while. You can check the progress of each of the transcoding tasks (one task for each resolution) in the Maintenance tab of the Admin area. If you see a task as ‘Stalled’, it means ffmpeg has either crashed, or that particular installation doesn’t like one (or more) of the parameters in the command sent to ffmpeg for transcoding.

It currently logs a decent amount of data to var/modules/transcode/log/transcode.log. This is the general output of the module behind the scenes. If a problem arises, this is the first place to look. Also, it logs ffmpeg’s output to a separate file (this is for 2 purposes, one so we can see what ffmpeg is doing having been executed from php/apache, and secondly so the module can work out how far into the transcoding process it is). Here, there’s one log file for each transcode/ffmpeg process.

If you can’t figure out why things aren’t working, drop me a line here, my Contact page, or on the Menalto forum here. I don’t expect this to be perfect, and it’s my first contribution to any open source project, so don’t shoot me down too hard 🙂

Dan

Download here: [download id=”27″ format=”1″]

Change Log
7th October 2010 :: Version 1 :: Initial release.
8th October 2010 :: Version 2 :: Removed dependency on php5-ffmpeg, refine audio codec selection list, add support for ffprobe to detect input video’s parameters.
10th October 2010 :: Version 3 :: Removed dependency on ffprobe, improved audio codec detection.
25th October 2010 :: Version 5 :: This module actually starts working on servers other than my own!
28th October 2010 :: Version 6 :: Fixed a bug that makes ffmpeg fall over if the input video’s audio sample resolution is not supported by ADPCM. It now fixes the sample rate to one of 11025 kHz, 22050 kHz or 44100 kHz (whichever is the next lower than the input sample rate).
30th October 2010 :: Version 7 :: Fixed issues providing an ffmpeg path and not saving correctly when no audio codecs are in the select list to begin with.
1st November 2010 :: Version 8 :: Bug fixes, thanks to netprince for the patch, and diverselab, jhend60, shecter and mamouneyya for help getting this working.
3rd November 2010 :: Version 9 :: Final stable release. Fixed issues with crashing due to ffmpeg not detecting/recognising the input file’s audio codec.
11th November 2010 :: Version 10 :: Bitrate tweaks and fix to audio bitrate channel detection and calculation.
13th September 2012 : Version 11 : Applied patch to prevent “Alternative Resolutions” banner appearing on photo pages and pages where transcoding has not been carried out (patch provided by Jannik Graversen).
15th September 2012 : Version 12 : Update codebase, cleanup, fix parsing of ffmpeg output to determine codec support. Utilise ffprobe to get details of source video as it provides a JSON interface. Added “Default Resolution” field, to auto-select on the frontend as opposed to the smallest resolution available. Only display “Alternative Resolutions” if 2 or more resolutions are available. Improvement on the back of patch from v11, using item->is_movie as opposed to counting resolutions to determine whether or not to display that area. Tested with ffmpeg version 0.6.5 through 0.11.1.
17th September 2012 : Version 13 : Fix versioning issue in installed, problem preventing save after upgrade or clean install.

Codex page on Gallery’s website: http://codex.gallery2.org/Gallery3:Modules:transcode

Join the Conversation

20 Comments

  1. This is really good…I’ve been thinking about upgrading to Gallery 3 and this has helped me figure out if I wanted to do it or not. Thanks also for the link to the transcode module

  2. Hi,

    The Transcode module for Gallery3 looks really interesting. I hacked on the transcode module for Gallery2 for a bit (adding basic support for encoding into Ogg Theora and WebM), and I’d love to have that functionality in this module as well.

    Couple of q’s

    1) Is the code in version control? (I’d be happy to help set that up)

    2) License? (GPLv2+ ? GPLv3+ ?)

    3) Based on my test install, the code currently targets FLV/Flash only — is that correct? (I didn’t see any method of selecting other output formats)

    Thanks for writing this module!
    –Q

  3. hi there,
    to answer your questions;
    1) yes, it’s in gallery3’s contrib github repository. it also has a place on my subversion server along with all my other projects.
    2) i don’t really know much about GPL licensing, which i know is odd for an open source developer. but i just write code that can help others. if anything, it’d be under the same license as G3 itself (GPL2).
    3) yes, that’s correct. this module will attempt to take any input video and convert it to flv video format with a compatible audio stream (aac prefered, or adpcm_swf if aac is unavailable). this is to keep the formatting consistent across all videos on your g3 installation, and to ensure compatibility with the built-in flv player. in future versions, i may also allow (as an option) the downloading of particular versions. in order to do this and know that it will work every time is if the output codec is the same across all videos. i suppose as webm becomes an accepted format and html5 starts being utilised more, this module may allow the encoding into flv and the use of the built-in flowplayer, or encoding to webm and the use of html5’s <video> tag.
    no probs 🙂
    dan

  4. Hi Dan,

    I tried the transcode module and encountered several issues:
    1. I cannot upload videos besides those accepted by Gallery3 anyway (flv, mp4, m4v). But e.g. renaming avi movies to an mp4 extension and uploading it works and the video is also transcoded.
    2. Firefox seems to have problems with the flowplayer. It only views the very first flowplayer instance. I can switch the resolution but as soon as I reload the page or want to view the next video, the flash area remains blank.

    Do you know of any workarounds?

    Regards,
    macin

  5. Hi,
    1. This is a known issue, however there is another version in the works which adds in similar functionality to that of the Server Add module. This will allow import of many different types of video files (avi, mov, etc). The downside to this is that you have to upload them to the server in a different manner (FTP, etc). However, this is a better method of transferring large video files, as it’ll reduce the strain on the webserver having to accept the upload, save it, and then go ahead on process the file. Your webhost would be very thankful! There is a “videos” module which has this Server Add modification built into it, but it disables the flowplayer flash and instead just allows you to download the original video file. My suggestion to you, and to various other people who’ve found this module, is to install it but only enable it to import videos with. Then disable it again in the modules screen to get the flowplayer back. The next version of transcode basically takes the upload portion of this module and modifies it so it’s a more elegant implementation.
    2. I’ve not heard of that issue to be honest. A bit more info might be helpful, such as what browser you’re using, version, operating system platform, etc.
    Dan

  6. 1. Any release plans yet? I am really looking forward to this feature.
    2. This was Firefox 3.6 on Linux. Updating the Gallery’s flowplayer files to the latest versions from flowplayer.org solved this issue.

    macin

  7. Hi.
    There are no release plans for transcode v11 as yet. It is still work in progress, however as I don’t have much spare time as it is, and with a few other modules on the go, it’s not an easy task to say the least! I’m in the middle of writing a Facebook integration module, and then the plan is to go back and finish/release transcode v11 and aws_s3 v3. If you’re subscribed to the thread on Gallery’s official forums, you’ll be one of the first to get updates to it, since I almost always post there before an imminent release.
    Watch this space 🙂
    Dan

  8. Hey, great module. I made some changes to it to allow me to upload my m2ts (AVHCD) videos to gallery 3. The module reads flv metadata for width and height which the m2ts videos do not have so I put in a few lines to get the info from ffmpeg-php. I also added a qscale setting in the settings page which bypasses the hardcoded bitrates for each resolution.
    I have tried the video module for downloading unsupported video types but cant get it to work. Once you modify the module so it sees the m2ts video, the module fails with:

    The file property does not exist in the Server_add_entry_Model class

    So the only way really is to rename the videos so they have an flv extension.
    I am looking forward to your next update.
    Thanks Again
    Chris

  9. Somehow I messed up my gallery3 install. I couldn’t figure out why I cant get transcode to work in Ubuntu lucid so I gave up. I decided to delete my gallery3 database in Centos and when I restarted Gallery, transcode doesn’t work anymore. I went to the original transcode-v10 and installed that and it still does not work in either Centos or Ubuntu. Everything else works. I am missing something very basic and specific to the transcode module. Maybe someone here could assist. Checking the /usr/share/gallery3/var/modules/transcode/log/transcode.log does not reveal much. The script enumerates the stdclass object and each field has seems like appropriate values but there is nothing else beyond that and no specific numbered transcode logs are generated. Any help would be appreciated.
    Thanks.
    Chris

  10. Never mind. Ignore my previous post. It must have been some permissions error. I reinstalled gallery3 and deleted my var folder and transcode is working again, even the modifications I put in. The only thing I cant figure out is why server add module insists on adding multiple copies of a file (it adds 2 extra versions with a numeric filename extension. So transcode module ends up encoding these as well. But thats not a transcode issue.
    Thanks
    Chris

  11. Love the module. Running into a small issue though…

    When transcoding, I am getting an error in the log of “sh: nice: not found”, which I assume means it can’t find the executable “nice”. It is installed in /usr/bin correctly.

    Do I need to change a path value somewhere to get that to work?

  12. Hi,
    Make sure that /usr/bin is in the PATH environment variable. Or symbolically link /bin/nice to /usr/bin/nice. Although, I’ve seen some setups where $_ENV[‘PATH’] is absolutely empty. This is the php.ini’s doing, and is somewhat out of control of the transcode module itself.
    I suppose in the future I could tweak the setup of this module to run a whereis or which on all the executables it needs and directly call them without relying on the environment finding it. In the meantime, do a phpinfo() script and have a look at $_ENV (near the bottom of the page) and see what PATH suggests.
    Dan

  13. Hard coding the path to nice worked to fix that error, and I get transcoding now. I’ll have to look into why it can’t find any of the /usr/bin files, and if there is a good secure way of allowing it.

    Also, something to note for everyone, if you upload a video that has a smaller height than one of your selected target sizes, it will skip transcoding the video for that other height because “there’s no advantage to it”. I was confused for a second on why I wasn’t getting a crappy 720p video for a 480p video I uploaded.

  14. Just a thought… I know you’ve sort of hard coded in the transcode to FLV, but have you considered the possibility of allowing the user to add multiple transcodes into different formats and resolutions as they desire?

    It would essentially just require keeping track of a list of the following information for each transcode:
    1. Video container
    2. Audio codec
    3. Resolution

    I know that flash video is the current standard, but as we move into HTML5 and mobile internet a bit more, it would be nice to be able to transcode into, say, WebM, FLV, and H.256, and then have Gallery display the right version for the current environment.

    As your module is the only one around that does this right now, I figured I’d ask if you have any plans for that… if not, would you be opposed to me hacking around to try and get that to work?

  15. Hello,

    I have installed transcode10.zip in G3 on a hosted server. I also manually installed ffmpeg 0.6 in my directory and set rights to 755.
    When I try to configure transcode settings and set the ffmpeg path to ../../ffmpeg (this is the path i put in the Admin advanced settings), it fails.

    I put the ffmpeg path, then click to verify ffmpeg path and it shows “Empty file path provided”.

    Not sure what I am doing wrong.

    Thanks for the great work on the module.

    Laurent

  16. Michael; I certainly don’t mind you hacking the module to enhance it. All that I ask is that you post your enhancements somewhere so I can include it in the main release. They sound like really useful enhancements, and I dare say that, at some point in the future, I’d end up doing that anyway!

    Laurent; Try set the path to ffmpeg as an absolute path. Chances are “pwd” is / (or DOCUMENT_ROOT). I can’t remember off the top of my head, but there may be a security element in there as well stopping you adding a “../..” type URL.

  17. Dan; thanks for the reply.

    I tried to put an absolute path like: /home/username/www/ffmpeg

    But same issue.

  18. hmm. i wonder if the slight variation in the code which does the ffmpeg verification has anything to do with it. i could do with a little more info from you, and possible access to your g3 installation (ftp) for 10 minutes. i have a theory, but is probably too technical to attempt to explain.

    use the contact me form here http://www.danneh.org/contact/ to send me an email where we can talk more.

    Dan

  19. I can’t get your module to detect any audio compressors. Are you going to update this or what? You have plenty of feedback for the audio not detecting, even someone supplying a fix.

  20. @Jimbo;

    Firstly, thanks for your interest in my module.

    I can only apologise for the radio silence as of late. When I started writing this module (and then released the first handful of versions), I had a fair bit of time on my hands, and needed something to keep my coding juices flowing. Lately, I’ve been inundated with work, and have barely had an hour to myself over the last 6-8 months or so.

    I share your frustration with the lack of updates on this. I have pending updates on my S3 module as well, and new stuff which I’ve written but not tested, including some bug fixes, which I need to release as well. But, as I don’t want to make the problem worse, I’ve been holding back until I’ve had enough time to spend sorting out the fixes to these modules.

    Especially so with the transcode module, as you quite rightly say it has issues detecting installed codecs. I’ve had contact from a few people who’s come across similar issues and contributed their code back to me when they’ve managed to find a working solution, which I shall be implementing into the official releases. ffmpeg has undergone a few version updates since transcode was first released, and I know that pretty much every time the output of that -codecs flag changes and breaks this extension, so will be making that a lot more robust.

    I promise, a fix is coming, and I should be in a position to launch something for testing in the next couple of weeks. As with anything open source, people like me contribute our time for free to make projects like G3 and their extensions. However, paid work has always taken a precedence, especially in these hard economic times. I’m having to work much more than I used to in order to make ends meet. It’s not just my extensions that have lost out with this spat of work I’ve had, my blog in general has too. So much to write about, and so little time to do it in!

    Thanks for your patience. And to anyone else out there who’s also waiting for a fix for this, thank you for your patience too. I promise, I haven’t forgotten. I haven’t discontinued development of it (or any other of my extensions, for that matter). I still intend to be active in the community, and launching more extensions in the future (I have a couple planned). I’ve just had a few more important things to deal with lately.

    Dan

Leave a comment

Your email address will not be published. Required fields are marked *