Showing posts with label MacPorts. Show all posts
Showing posts with label MacPorts. Show all posts


mplayer vs. polish subtitles

Typically, when I start seeing a film, I only have enough time to quickly get some subtitles, try to make them more-or-less work with the film at hand and ... that's it.
So, lots of repetitive, rushed fixes but no long-term solutions.

But this time I got sick of it and tried to get to understand the problem. Which is: Polish subtitles don't work with mplayer, or at least not with the mplayer built with MacPorts' mplayer-devel port, which uses mplayer's SVN HEAD.

The option -subcp cp1250 does select the codepage (cp1250 is variously called Windows Latin 2 or Windows Central European encoding, which seems to be the typical encoding used by polish subtitles on the net)

The option -subcp enca should auto-detect the encoding, but the port disables enca at configure time, and provides no way to enable it. I'll try to send a patch for that.
In the meantime, enca -L pl -i file by itself works nicely (enca is provided by a port). For difficult cases when enca fails, the chardet module for python should work; didn't try it yet.

So now we have subtitles with the right encoding. Next problem: the displayed subtitles have some polish letters missing! (not all, which is strange. ć and ł do work, but ż and ą don't)

-fontconfig, -font don't seem to do anything. -font in particular doesn't seem to mind what kind of file I feed it, or even if it exists, and -msglevel all=9 doesn't show anything. Looks like -nofontconfig is needed for -font to start having any effect. Which sounds logical given that the fontconfig project seems to be all about autoconfiguring font management, but the mplayer docs say pretty little about all of this.
So, with -nofontconfig present, -font does accept .ttf files, but also .desc files.

(I have a pack of fonts for mplayer called font-arial-cp1250, which contains some variations of some arial font consisting on sets of .raw files with a main font.desc file. I seem to remember that I downloaded it from some forum. The files are dated from 2003. The font.desc files can be feed to mplayer with the -font option. And they don't work, or rather only seem to work when the encoding is NOT correctly selected with the -subcp option. So this is a dead end, probably outevolved by mplayer in these last 8 years. So better forget this.)

My ~/.mplayer directory also contains a subfont.ttf file, which seems to work fine with the -font option. I have tried a couple of other fonts and they seem to lack the polish characters, so not every .ttf will be OK.

(I don't know if that subfont.ttf is standard. This .mplayer subdirectory has probably followed me through maybe 4 or 5 OS X versions, 4 computers and 2 architectures. Which means, no idea where it came from. The current mplayer port doesn't seem to include such a thing, which is to be expected since MacPorts makes some effort to install things only in well defined places. But I remember having played with other versions of mplayer, from fink to some binaries. Who knows.)

So, to summarize: with -subcp goodencoding -nofontconfig -font goodfont.ttf we should be OK.

And yet, it can be better. At some moment I discovered that the -ass option works. -ass uses libass, which is not covered by any port, so I didn't expect it to work; but mplayer seems to have its own internal version of the library (seen at the configure stage with something like port install -d mplayer-devel, or in the configure.log if macports has not been configured to --clean after installation). And -ass is amazing. -nofontconfig nor -font again don't seem to work, and I don't know where mplayer is getting its fonts now, but it is a good font with all the polish characters. And not only that, but configuration commands contained in some subtitles (.txt files, with not only the subtitles but commands that look like {y:b}{c:$0000ff}; RTF? CSS?) do work, so instead of the occasional rubbish now the subtitles render beatifully, with colors and bolds and italics, oh my.

So, to re-summarize: the best option is something like mplayer vidfile.avi -sub subfile.txt -subcp cp1250 -ass  
(and if I manage to send the patch for enabling enca, it should be something like -subcp enca:pl:cp1250)

Keeping in mind that all of this is for MacPorts' mplayer-devel, which builds the SVN head. So all of this might be quite temporal. Which sucks. Hard.

(Why not use VLC and forget about all of this? Because VLC allows very little control, and most any change means having to stop and start again. That's OK if the film and subtitles are perfectly matched, but that's not usually my case. Meanwhile, in mplayer one can tune lots of things, even while watching the film: go forward and backward in the subtitles and synchronise them to the video / sound; set say 90% speed (VLC only allows 66%, 50%, ...); move subtitles on the screen, even render the film with more black space on the bottom should one want the subtitles to not overlap the image. And then of course is mencoder...)


tcpflow 1.0.2 doesn't work with net expressions

Tcpflow 1.0.2, as built with MacPorts, doesn't work when net expressions are used.
But 1.0.6 does work.

I already submitted a new portfile so it should be available soon.


Building socat in OS X 10.7 Lion

socat ( as of this writing) doesn't compile with the clang, the standard compiler in Mac OS X 10.7 Lion (10.7.2 as of this writing). It does compile if instead of clang one uses for example llvm-gcc-4.2.2.

The developer reports that this is a bug; only gcc is supported in socat, but there are compatibility fallbacks for other compilers. Only, the fallback was missing on the file that fails to compile, xioexit.c. The fix is easy:
@@ -5,6 +5,7 @@
/* this file contains the source for the extended exit function */

#include "xiosysincludes.h"
+#include "compat.h"
#include "xio.h"
(if someone is trying to build something like socat, I guess he doesn't need help about patchfiles)

This problem was also present in MacPorts' port for socat. I have already reported it and provided a new working portfile, so I guess it won't be long until it is published.


Directory permissions vs local Portfiles in MacPorts

~/MacPorts/ports$ sudo port install socat
--->  Computing dependencies for socat
could not read "/Users/mija/MacPorts/ports/sysutils/socat/Portfile": permission denied

I was having strange problems to use a locally edited portfile. Turns out the permissions were wrong in a directory in the path; each of the directories should have at least o+rx permissions, and strangely my $HOME had none of those (strange because other users in my computer had o+rx, admins or not).

Note that MacPorts lately (from 2.0?) has started to use the user nobody at some points of its workings, and root at others; in this case the user nobody was the one unable to reach the Portfile. A way to check what this user sees is "sudo -u nobody ls -leda@ /Users/mija/blabla".

A workaround is setting macportsuser in /opt/local/etc/macports/macports.conf to root, but that's not a good idea. MacPorts is doing the the sensible thing, de-escalating privileges when they are not needed; that way, if something went awry, the problems should be much smaller. Lion is doing a lot to be secure, let's keep it that way.

So fix those permissions, goddamit.


Compiling Transcode 1.1.6 and 1.2 in OS X

(express post, too much time spent on this but too much work spent to let it slip without benefitting someone else)

1.1.6 is not released, had to be hg-cloned. In fact, the repository looks like the 1.1.6 tag was at some moment added but then deleted?? That branch also has the latest video stabilization, no idea yet if it is also in 1.2

Can be compiled. Get the hg repository via hg itself, not one of the zip/bz/etc packaged from the website since the compilation scripts expect to find the .hg files. Read the README and INSTALL, you'll need to create the configure yourself (easy, just read)

I am using MacPorts, and even installed the included Transcode 1.1.5, so all the requirements are already installed. Most of the expected work was about configuring tne non-macports transcode to use the /opt dirtree.

(Incidentally: 1.1.5 has horribly mismatched documentation, I have had to resort to using strings and source code to understand what some of the utils were supposed to be doing and how. The wiki doesn´t help. Also, not really working some of the combinations of export module/codec blow up without saying anything, no idea if that's because of transcode itself or because of lame, ffmpeg versions, who knows. Also, seems to be accepted that mp4 container files can't be generated directly by Transcode. I am interested on using the video stabilization plugin, but I if I had known that the time invested would be this much, ...)

1.2's generation of configure uses a script named, which doesn't work right and causes problems: the generated configure fails with
./configure: line 4: .: filename argument required
.: usage: . filename [arguments]
Inspection of the configure source shows that it was badly generated. uses echo -n blabla trying to output a "blabla" with no line feed. But echo in sh doesn't work like that, so the actual output is "-n blabla\n". (mhm, no, both sh and bash use /bin/echo but it behaves differently, interesting). Anyway: that's nasty, didn't anyone check it? And the errors it causes are insidious, I wasn't sure if those were status messages, warnings or plain errors. The easy fix is change the echo -n $1$HGVER for printf $1$HGVER .

Even after having fixed that, 2 telltale failed output strings remained (something like warning: AC_INIT: not a literal: -n 1.2.0-0a9d7c95b961
). But I could not find where they were coming from. No, was not it, and I could not find any file which had cached the string or parts of it. Before the fix in there were about 6 of those, after there were 2. Anyway, after some infructuous searching, I decided to try to go on... at least configure now worked and nothing exploded in a seemingly related way.

The ./configure line I used:
./configure --prefix=/opt/local --enable-libavcodec --enable-ffmpeg --enable-experimental --enable-statbuffer --enable-lame   --enable-xvid   --enable-x264   --enable-libquicktime   --enable-faac    --enable-libavformat --with-lame-prefix=/opt/local --with-lame-includes=/opt/local/include --with-xvid-prefix=/opt/local --with-xvid-includes=/opt/local/include --with-xvid-libs=/opt/local/lib --with-faac-prefix=/opt/local/bin --with-faac-includes=/opt/local/include --with-faac-libs=/opt/local/lib --disable-libdvdread --disable-libjpeg 

The xvid support is half-baked. I had to
export CC=I/opt/local/include
so the xvid support finally worked, since sometimes the --with-xvid-includes was left unused by the Makefile.

Incidentally, I think that also helped x264.

Transcode 1.2 has been seemingly abandoned for about 18 months now. The last post in the website says it should be about alpha quality, and that they were desperately looking for new developers...

The make failed because in a number of submakefiles the tcmodule libs  path was not being added to the linker. I fixed it by adding $(LIBTCMODULE_LIBS) whenever the linker failed (with errors like:
Undefined symbols:
  "_tc_module_info_match", referenced from:
      _tc_module_match in tcmp3cut.o
  "_tc_module_info_log", referenced from:
      _tc_module_show_info in tcmp3cut.o
ld: symbol(s) not found
collect2: ld returned 1 exit status
make[2]: *** [tcmp3cut-1.2] Error 1


(note the "_tc_module_info_match": "tc_module_info_match" is a function in the "tcmodule-info" file in the "tcmodule" directory (sometimes Spotlight is useful! ;P). Then, note the "tcmp3cut": that was the part being compiled. Looking for it in the makefile showed that it was not including the $(LIBTCMODULE_LIBS), which was defined in the beginning of the file, but not being used as much as the rest of definitions)

The tools directory had worse problems. The tcstub.c file is used when compiling other files, and 4 definitions conflict with libtc.a. Since libtc  seems to be generic for all of Transcode and the stub seems to be less important, I guessed the stub versions of the definitions had to go. I commented them out (tc_new_video_frame, tc_new_audio_frame, tc_del_video_frame, tc_del_audio_frame).
The errors were like this:
ld: duplicate symbol _tc_new_video_frame in ../libtc/.libs/libtc.a(tcframes.o) and tcmodchain-1.2-tcstub.o

With that, it finally compiled.
Beware, make install is supposed to install in parallel to existing 1.1-line transcodes (the new files should have "-1.2" added to their names), but some files had the same name and got overwriten. Example: avifix.
Which is doubly insidious because the tools directory had those stub file problems, which make all the directory contents suspicious...

When I have a moment to check / compare the 1.1.5, 1.1.6 and 1.2, I'll try to leave a note.


Bus error in MacPorts' Python

PyQt4 (MacPorts' py25-pyqt4) was causing a Bus Error.

Some previous problems with the Python 2.5 installed by MacPorts made me think it could be some dependency with the native library called by the Python module; that is what I think that was also happening with py25-hashlib, which caused an error about being unable to find _md5 or some such, and which got fixed by uninstalling MacPorts' openssl 0.9.8g and installing 0.9.8h (and reinstalling py25-hashlib afterwards).

(and I say "think" because maybe that problem was unrelated to openssl, and in fact could be related to python_select, which I maybe had or had not run. Read on for more on that. Anyway, there was also some report online of problems between Pythons' hashlib and openssl because of the exact version used, and mentioned Python being a bit fragile in that respect — although it didn't say if that was because of a not-too-robust build/installation process, or because of Python's design.)

But here the problem was different. Reinstalling Qt 4 and PyQt4 did not help. After some googling, I found someone (thanks jherm) with the same problem... and who in some chat log is told by some MacPorts developer (?) the anticlimactical solution: follow the instructions at the end of the installation of MacPorts' Python 2.5. That is,
$ sudo port install python_select $ sudo python_select python25
Looks like this has to be run "to complete the installation". Sheesh. So that was a hard requisite? Shouldn't that have been told pretty explicitly then? (I'm seeing a number of reports about this kind of problem, so yes, I think the warning should be quite more explicit... I should start some bug reporting, I guess.)

So the problem seems to be that PyQt4 uses Python in its build process, assuming that it is the same in which it will be run afterwards. But in my case I had not run python_select after installing MacPorts' python25, because it seemed optional, and I didn't want to use it as the default… so PyQt4 was building against OS X's default Python.

So, after python_select, PyQt4 builds OK. It doesn't manage to get fully installed, mind you, but I guess that this other problem has more to do with MacPorts. Anyway, with a
$ sudo port -f install py25-pyqt4
it finally gets installed (although something remains to be finished, like the MacPorts' database getting properly updated; that can be checked with -d).

All of this was in the process to try to make Anki run from source. I was trying to test some modifications before sending the patches, but making sense out of the tortuous source and setting up the environment have required much more time than expected. I should be studying Polish instead! The exam is coming! :P