EIT (Event Information Table) packets overview - ALPHA Phase: investigation and proof of concept only!!!

Hi guys,

I am trying to begin a new thread about EIT packets so that we can get a better view about where to find names of contents, based on the assumption that these packets are saved within a recording by the LG (thanks Sebastien for the info).

We have to dig in some standards to trace our way through all these bytes...
MPEG-2 TS structure is ISO/IEC 13818-1 ... I think you should pay to access it but I find a link about it:
http://neuron2.net/library/mpeg2/iso13818-1.pdf (have a look at p.18 2.4.3.2 ... if the CRC are removed by the LG, otherwise it's gonna be much harder)
structure of TS packet is in table 2-2

DVB signalling is ETSI EN 300 468 (free from their website)
From there,
EIT PID is 0x0012
EIT actual current/following table_id is 0x4E
Structure of EIT is p. 25 in table 7 of v1.11.1 (2010-04 version, latest one I think)

So we should look for an hexadecimal string that begins by (for unknown values, so far, x=bit, X=1/2byte; []=binary, otherwise hexadecimal notation):
0x47[xxx00000]12XX....{EIT section} where EIT section begins by:
0x4E...

=> still have to check if adaptation field is used, and then about EIT section itsef, how to get access to event name...

Cheers,
Matt

woww, impressed
thanks!! I will take a look at this later. thanks.

I am browsing (hexa) a TS file I got from a USB key (have no access to my LG, but I believe it should be the looking the same).

I should be able to find EIT current/following by searching the following hexadecimal heading string (where X is a continuity counter):
0x4700121X4E
Then, first X is of no importance (so far), but following 3 (12 bit field) describe the length of the section...

The example I found so far (don't know yet whether it is too short or not ):

Código:

470012144E06006672650000006AD85310284100245181F55024F1010066726505766964656F2C20343A332061737065637420726174696F2C203235487A5504667261014D5F6672652D05424149452044455320464C414D424F59414E5453202D20534149534F4E20332022455049534F4445203130222D05424149452044455320464C414D424F59414E5453202D20534149534F4E20332022455049534F4445203130224EFB0166726500F50543485249535449414E2C20706572470DC412A731F01792F13B8101E7C003FCF0075559B9D7139C5E6BA2B4CCDBE2805F0DCB6571538A38F6F0E3957ACB2EF541F4E21812D0E80FDD39AAE74F0D3E4770EB3E9C6CBCB9EF09A56421175C24C37C70994DF7C7EA70E42408539C55370B828E78229CD002C03707E13032FFFC14003B800795C3503E0D800300F713921F0757710022CA0A805D42387A8C581D82C3E4AC000000000001053A5734ECAFBCFB3AF7CC03E3E00FE0019404C0D4B27E1234CEBC1D1C16380D8FBD470DC11CCF587ED518027013814C00560300D013A03802A26001B80DF070C6C4F802F218217FD1340CF22E800C007619C3002F2C0322F89C0077D29C14AF00663096061B0764236D210741A02F7A8534E3E31A6569E31282CA48C2D4DD0C9475C00D804C03AC018930060057130303501A86C1A580ED00376249209BC0C12D069DDB400DC0AFE432100800604C2897C9BF0C4F0281A184326B8C5AD1C34210701E763E03034A2C02A0D4A521A02846C181890DE924AC6842123CD08D470DC3136312CEDD811401B0697CF3949199FABE2D2022963061E011A47AB2FAD6B103BAE4FE30690320700ADD42143EEAE4D417B6ECEE022400DDB1A792F381222798FE443EDF00A1C9F901

So breaking it gives:
0x470012144E TS header + PID EIT actual current/following table_id
0x0 => (weird, I believed there should be a 1 at the beginning ...??)
0x600 => size of section below i.e 1536 octets are following (by pair of letters, looks too short now ^^)
0x6672 => service_id, i.e should correspond to one of the tv that is actually transported over the multiplex. Hopefully, LG only keeps EIT related to the recorded program (otherwise, we will have to read other things ... like NIT and SDT ... bad)
0x65 => 01(?) 10010(version number) 1(applicable => OK)
0x00 => section number (00 is first one)
0x00 => last section number (we got everything in that packet)
0x006A => transport stream ID about which this EIT gives information
0xD853 => should be 0x20FA for french DTT ... shit, got a bad example (just checked, that's a local multiplex transmitted over Paris ... I should have taken a national one ...)

Ok, back later to try again on an other one ...

Very interesting.
In your example we can see 4 frames of TS,
each frame is lenght 188 dec, as states the spec.
And the first byte (sync. ) is alwasy 0x47.

Código:


470012144E06006672650000006AD85310284100245181F55024F1010066726505766964656F2C20343A332061737065637420726174696F2C203235487A5504667261014D5F6672652D05424149452044455320464C414D424F59414E5453202D20534149534F4E20332022455049534F4445203130222D05424149452044455320464C414D424F59414E5453202D20534149534F4E20332022455049534F4445203130224EFB0166726500F50543485249535449414E2C20706572
470DC412A731F01792F13B8101E7C003FCF0075559B9D7139C5E6BA2B4CCDBE2805F0DCB6571538A38F6F0E3957ACB2EF541F4E21812D0E80FDD39AAE74F0D3E4770EB3E9C6CBCB9EF09A56421175C24C37C70994DF7C7EA70E42408539C55370B828E78229CD002C03707E13032FFFC14003B800795C3503E0D800300F713921F0757710022CA0A805D42387A8C581D82C3E4AC000000000001053A5734ECAFBCFB3AF7CC03E3E00FE0019404C0D4B27E1234CEBC1D1C16380D8FBD
470DC11CCF587ED518027013814C00560300D013A03802A26001B80DF070C6C4F802F218217FD1340CF22E800C007619C3002F2C0322F89C0077D29C14AF00663096061B0764236D210741A02F7A8534E3E31A6569E31282CA48C2D4DD0C9475C00D804C03AC018930060057130303501A86C1A580ED00376249209BC0C12D069DDB400DC0AFE432100800604C2897C9BF0C4F0281A184326B8C5AD1C34210701E763E03034A2C02A0D4A521A02846C181890DE924AC6842123CD08D
470DC3136312CEDD811401B0697CF3949199FABE2D2022963061E011A47AB2FAD6B103BAE4FE30690320700ADD42143EEAE4D417B6ECEE022400DDB1A792F381222798FE443EDF00A1C9F901


Indeed, but what you are interested in are not all the TS packets but only these transporting name of current program, and these should be:
0x470012XX4E packets
At least, according to the standards I am reading, if I am not wrong.
The issue is that I can not find these packets on French DTT ... wondering why, but I will investigate.

Cheers,
Matt

Very ugly perl script to locate hexadecimal offset of these bloody EIT actual sections of p/f programs within a TS file (keep running under 100Mo so far ):

Código:

#!/usr/bin/perl
my ($a,$b);
my $cpt = 0;

my $temp = 1 + $#ARGV;
print "Arguments fournis en ligne de commande: $temp\n\n";

if (!(($#ARGV == 1) or ($#ARGV == 3)))
   {
   print "Arguments: FSOURCE Racine_FRES [xxxxx yyyyyy]\n";
   print "FSOURCE: nom du fichier TS source\n";
   print "Racine_FRES: racine du nom des fichiers résultats (autant que d'EIT...)\n";
        print "xxxxx: rfu_tbd\n";
        print "yyyyy: rfu_tbd\n";
        die ("Nombre d'arguments incorrect");
   }
else {
   $a=$ARGV[0];
   print "FSOURCE: $a\n";
   $b=$ARGV[1];
   print "Racine_FRES: $b\n";
        if ($#ARGV == 3)
           {
            }
   }

open (FSOURCE,"<$a") or die("open: $!");
binmode FSOURCE;
my $octet;
my $hex, $hex1, $hex2, $hex4;
while (read(FSOURCE,$octet,1))
   {
    $hex = unpack 'H*', $octet;
   $cpt ++;
   if ($hex == 47)
      {
      read(FSOURCE,$octet,1); # normalement 00
      $hex1 = unpack 'H*', $octet;
      read(FSOURCE,$octet,1);   # vérifier si PID 12 pour EIT
      $hex2 = unpack 'H*', $octet;
      read(FSOURCE,$octet,1);   # vérifier compteur de continuité
      $hex3 = unpack 'H*', $octet;
      read(FSOURCE,$octet,1);   # vérifier si table_id est bien 4e
      $hex0 = unpack 'H*', $octet;
      $cpt += 4;
      if (($hex1 eq '00') && ($hex2  eq '12') && ($hex0 eq '4e'))
         {
         my $convhex = sprintf("%x", $cpt);
         print "position: $cpt (0x$convhex)\n";
         }
      }
    }

close (FSOURCE);


die ("Opération terminée");

Use it with following syntax:
perl.exe code.pl source_file blablaorwhateveryouwant > result.txt

Hi,
i have created the perl file in linux,
and executed with a short file Gormitis.ts
5 seconds saved from tv

[victor@asus ts]$ ./ts.pl ./Gormitis.ts
Arguments fournis en ligne de commande: 1

Arguments: FSOURCE Racine_FRES [xxxxx yyyyyy]
FSOURCE: nom du fichier TS source
Racine_FRES: racine du nom des fichiers résultats (autant que d'EIT...)
xxxxx: rfu_tbd
yyyyy: rfu_tbd
Nombre d'arguments incorrect at ./ts.pl line 15.
[victor@asus ts]$

what are the arguments? the offsets... ?

abt

wow
I have found several 47 00 12 packets,
just a short portion

Código:


47 00 12 17  72 6F 67 72 | ...ëyÀ¯oo¸.ÐG...rogr
0002:FB0C | 61 6D 61 63  69 6F 6E 00  29 E4 D8 C8  10 03 12 00  26 48 01 59 | amacion.)äØÈ....&H.Y
0002:FB20 | 50 08 01 01  00 73 70 61  05 20 50 08  02 03 00 73  70 61 05 20 | P....spa. P....spa.
0002:FB34 | 55 04 65 73  70 04 54 02  80 80 4D 15  73 70 61 10  05 47 75 67 | U.esp.T...M.spa..Gug
0002:FB48 | 67 65 6E 68  65 69 6D 20  32 30 31 30  00 4E FF 01  73 70 61 00 | genheim 2010.Nÿ.spa.
0002:FB5C | F9 05 4E FA  6D 2E 20 63  61 70 2E 3A  20 33 3B 20  48 65 6E 72 | ù.Núm. cap.: 3; Henr
0002:FB70 | 79 20 52 6F  75 73 73 65  61 75 20 65  73 20 75 6E  61 20 65 78 | y Rousseau es una ex
0002:FB84 | 70 6F 73 69  63 69 F3 6E  20 63 6F 6E  20 6C 61 20  71 75 65 20 | posición con la que
0002:FB98 | 65 6C 20 4D  75 73 65 6F  20 47 75 67  67 65 6E 68  65 69 6D 20 | el Museo Guggenheim
0002:FBAC | 42 69 6C 62  61 6F 20 72  69 6E 64 65  20 68 6F 6D  65 6E 61 6A | Bilbao rinde homenaj
0002:FBC0 | 47 05 DD 18  94 01 19 B6  49 42 5B E4  40 0E 30 10  7F 88 01 98 | G.Ý....¶IB[ä@.0.....
0002:FBD4 | 03 EF 60 08  74 59 8F EB  06 00 E4 A5  C6 BE A1 04  4F 0E A0 03 | .ï`.tY.ë..ä¥Æ¾¡.O. .
0002:FBE8 | 56 CB 80 42  FF 22 21 1D  F0 02 0F 35  6D D5 68 D6  0B D0 03 2E | VË.Bÿ"!.ð..5mÕhÖ.Ð..
0002:FBFC | C5 47 C0 AE  49 C4 EE 60  24 7F E8 71  6D 53 7A 4E  A4 FE 37 29 | ÅGÀ®IÄî`$.èqmSzN¤þ7)
0002:FC10 | 0E 36 74 AB  1B 46 8D 1A  35 32 6D 18  B0 62 C9 86  D0 07 26 00 | .6t«.F..52m.°bÉ.Ð.&.
0002:FC24 | 0C 4C 24 A1  25 B0 A8 20  81 B0 20 80  E0 26 80 29  04 13 7F E8 | .L$¡%°¨ .° .à&.)...è
0002:FC38 | D6 DC 03 93  00 06 26 00  0C 51 B0 82  18 19 82 08  0D 82 68 03 | ÖÜ....&..Q°.......h.
0002:FC4C | 13 C1 37 FE  D5 B9 81 0C  06 7C C1 37  FF 51 B6 04  3F F1 27 01 | .Á7þÕ¹...|Á7ÿQ¶.?ñ'.
0002:FC60 | ED A8 04 3F  DE 04 10 19  04 DF F0 00  A8 00 62 07  37 60 10 C0 | í¨.?Þ....ßð.¨.b.7`.À
0002:FC74 | 34 00 F0 13  40 1C 9C 00  47 05 DF 3B  9F 00 FF FF  FF FF FF FF | 4.ð.@...G.ß;..ÿÿÿÿÿÿ
0002:FC88 | FF FF FF FF  FF FF FF FF  FF FF FF FF  FF F


vic1972 escribió:

[victor@asus ts]$ ./ts.pl ./Gormitis.ts
Arguments fournis en ligne de commande: 1
.....

what are the arguments? the offsets... ?

Hi Vic,

I am really sorry, it is a really ugly piece of code I did not finish (I copied it from an other code and did not finish it).
Forget about the last two arguments, they are of no use.
Actually, only the first one is meaningful, but the second one is needed but not used yet.
So you have to call it that way:

./ts.pl ./Gormitis.ts uselessstuff

And it will search for the pattern 0x470012XX4E which is meant, according to the standard, to be the EIT related to the programs really sent over that TS now or soon (other value than 4E should means they are either programmed on the TS but later, or part of other TS ... I am not sure that all broadcaster stick to the standard anyway ...) and give you the offset of the data after the 4E byte.

I used it on a windows PC ... you could need to change the H of the unpack method towards a h due to Little/Big Endian stuff.

Cheers,
Matt

I think I made a mistake. EIT are spread over more than one TS packet. We should first target getting the first one, and it is probably not a 0x4700121X4E packet but rather a 0x4740121X4E packet... My script should be adapted therefore. The following packets (00 rather than 40) have probably a limited header, and I was therefore trying to find things in an inexistent header...

Moreover, I am afraid the script won't work correctly on linux devices (due to endianess), which is not an absolute issue since this is a bit of a rubbish code and since the goal is just to locate the right EIT and validate the search/pattern principle so that we can later on think of a better program (that's where I will probably stop ... as you could see, my perl is already ugly, so I let you imagine my C xD )

I'll tell you later if I do some progress.

Cheers,
Matt

Thanks
I have already started my own C program

Very interesting all this staff, I can even see the encoded channels over TDT premium.

Either my script is absolutely wrong (could be), either there is an other issue because I can not find table_id 4e over 5 minutes recording of the R2 mux in Paris ...
I'll check on an other one ... wondering how much they stick to the standard...

For your own dev, you can probably find useful things into the C DVB libraries.
Cheers,
Matt

I got rid of old script ...

Here is a new simpler one. It extracts all TS packets (188 bytes) that match the EIT actual present/following new section scheme, i.e. 0x4740121X004E beginning and print it as an hexdump.
Next step is to extract the useful data from it (hex to ascii conversion at later stage).

Please note that it is running ok on windows and that you could need to change the H to h in the unpack method if you run it on linux...

Still ugly code, but better than previous one xD

Cheers,
Matt

Código:

#!/usr/bin/perl
my ($a);
my $cpt = 0;

my $temp = 1 + $#ARGV;
print "Number of parameters: $temp\n\n";

if (!($#ARGV == 0))
   {
   print "Parameters: FSOURCE\n";
   print "FSOURCE: TS file name to search in about EIT actual present/following\n";
   print "In case of specific characters, please use quote around the name of the file\n\n";
   print "This script digs in a TS file, looks for EIT actual present/following\n";
        print "and then print the corresponding TS packet in hexdump form\n\n";
        die ("Nombre d'arguments incorrect");
   }
else {
   $a=$ARGV[0];
   print "provided FSOURCE is: $a\n";
   }

open (FSOURCE,"<$a") or die("open: $!");
binmode FSOURCE;
my $buffer;
my @tspacket;

while (read (FSOURCE, $buffer, 188))
   {
   @tspacket = split( //, unpack 'H*',$buffer);
   if (($tspacket[0]!=4) or ($tspacket[1]!=7)) {die("Corrupted file source or endianess issue (please then replace H by h in perl script)\n");}
   if (($tspacket[2]==4) and ($tspacket[3]==0) and ($tspacket[4]==1) and ($tspacket[5]==2)) # first EIT
      {
      if (($tspacket[6]==1) and ($tspacket[10]==4) and ($tspacket[11] eq 'e')) { print @tspacket,"\n";}
      }
   }

close (FSOURCE);

die ("End of operation\n");

I'm sorry I involve here, but we have no perl and I would never compile perl for MS450 because we really don't need it and I have more ideas how to fill-up next 40MB instead install the Perl (despite the fact we have only 20MB free now).

it's ok,
I have already started a parser in c languague.

I'm clear of that. It came in mind right after I send this message
Sorry, I'm feeling overworked now

Hi Keltek,

I am not expecting to get perl ported to the LG, don't worry, but that's the only langage I can use to make some investigation (because it is "easier" than C). I am only performing investigation here, to try to find where things are. If there is no need anymore of such "proto" investigation, let me know, I won't code further
If I can then make things clear enough so that we (actually one of you) can get benefit from that in order to develop whatever you want, I would be happy even if I would prefer directly to code in C, compile for mipsel, etc. but I am afraid I really can not do that, so my perl script remains the best way I can try to help, if that is of any use.

Cheers,
Matt

Latest version of perl script, for those who want.
It identifies ALL the right EITs (should be the right ones, at least) TS packets (first TS packet only, beware in case of EIT split over more than one TS packet) within a TS file, and print data about it structures. Next step is to identify the descriptors within the last loop (descriptors_loop), i.e the short_event, etc. which contains the name of the current/following event (i.e. what we want) and then to chose one among all the available one to modify the name of the recording, before actual last step: port that, if feasible, over the LG without perl (and I am really sorry not to be able to C code).

So far it runs on XP SP3 with Strawberry perl (beware of endianess if you try it over unix or linux)

The goal is just to better understand these packets, in no case to get it ported as perl script over the LG ... and use/modify it at your own risk.

Código:


#!/usr/bin/perl
my ($a);
my $cpt = 0;

my $temp = 1 + $#ARGV;
print "Number of parameters: $temp\n\n";

if (!($#ARGV == 0))
   {
   print "Parameters: FSOURCE\n";
   print "FSOURCE: TS file name to search in about EIT actual present/following\n";
   print "In case of specific characters, please use quote around the name of the file\n\n";
   print "This script digs in a TS file, looks for EIT actual present/following\n";
        print "and then print the corresponding TS packet in hexdump form\n\n";
        die ("Nombre d'arguments incorrect");
   }
else {
   $a=$ARGV[0];
   print "provided FSOURCE is: $a\n";
   }

open (FSOURCE,"<$a") or die("open: $!");
binmode FSOURCE;
my $buffer;
my @tspacket;

while (read (FSOURCE, $buffer, 188))
   {
   @tspacket = unpack 'C*',$buffer;
   if (!($tspacket[0]==0x47)) {die("Corrupted file source or endianess issue (please then replace H by h, C by c, etc. in perl script)\n");}
   if (($tspacket[1]==0x40) and ($tspacket[2]==0x12)) # first section of an EIT
      {
      if (($tspacket[3]>=0x10) and ($tspacket[3]<0x20) and ($tspacket[5]==0x4e)) #check whether this is an EIT actual present/following
         {
         my $section_syntax_indicator = (0b1000_0000 & $tspacket[6]) / 0b1000_0000;
         my $section_length = (0b0000_1111 & $tspacket[6])*0x100 + $tspacket[7];
         my $service_id = $tspacket[8] * 0x100 + $tspacket[9];
         my $version_number = ($tspacket[10] & 0b0011_1110) / 0b0000_0010;
         my $current_next_indicator = $tspacket[10] & 0b0000_0001;
         my $section_number = $tspacket[11];
         my $last_section_number = $tspacket[12];
         my $transport_stream_id = $tspacket[13] * 0x100 + $tspacket[14];
         my $original_network_id = $tspacket[15] * 0x100 + $tspacket[16];
         my $segment_last_section_number = $tspacket[17];
         my $last_table_id = $tspacket[18];
         my $descripteurshexa = "";
         my $descripteurs = "";
         my $result = "section_syntax_indicator: " . sprintf ("%1u",$section_syntax_indicator) . "\n" .
         "section_length: " . sprintf ("%4u",$section_length) . "\n" .
         "service_id: 0x" . sprintf ("%04X", $service_id) . "\n".
         "version_number: ".sprintf ("%02u", $version_number)."\n".
         "current_next_indicator: " . sprintf ("%1u",$current_next_indicator)."\n".
         "section_number: 0x". sprintf ("%02X", $section_number) . "\n".
         "last_section_number: 0x". sprintf ("%02X", $last_section_number) . "\n".
         "transport_stream_id: ".sprintf ("%04u", $transport_stream_id)." (0x".sprintf ("%04X", $transport_stream_id).")\n".
         "original_network_id: ".sprintf ("%04u", $original_network_id)." (0x".sprintf ("%04X", $original_network_id).")\n".
         "segment_last_section_number: 0x". sprintf ("%02X", $segment_last_section_number) . "\n".
         "last_table_id: 0x". sprintf ("%02X", $last_table_id)."\n";
         my $compteur = 19;
         while (($compteur < 188) or ($tspacket[$compteur] == 0xFF))
            {
            my $event_id = $tspacket[$compteur] * 0x100 + $tspacket[$compteur+1];
            my $start_time = ((($tspacket[$compteur+2] * 0x100 + $tspacket[$compteur+3])*0x100 + $tspacket[$compteur+4])*0x100 + $tspacket[$compteur+5])*0x100 + $tspacket[$compteur+6];
            my $start_timeMJD = ($tspacket[$compteur+2] * 0x100 + $tspacket[$compteur+3]);
            my $start_timeBCD = (($tspacket[$compteur+4])*0x100 + $tspacket[$compteur+5])*0x100 + $tspacket[$compteur+6];
            my $duration = (($tspacket[$compteur+7] * 0x100 + $tspacket[$compteur+8])*0x100 + $tspacket[$compteur+9]);
            my $running_status = (0b1110_0000 & $tspacket[$compteur+10]) / 0b0010_0000;
            my $free_CA_mode = (0b0001_0000 & $tspacket[$compteur+10]) / 0b0001_0000;
            my $descriptors_loop_length = (0b0000_1111 & $tspacket[$compteur+10]) * 0x100 + $tspacket[$compteur+11]; # BIG warning there, it can goes over the current TS packet, be ready to fill with zero ...
            
            my $event = "event_id: 0x". sprintf ("%04X", $event_id)."\n".
            "start_time: 0x". sprintf ("%010X", $start_time)."\n".
            "duration: ". sprintf ("%u", $duration)."\n".
            "running_status: ".sprintf ("%u", $running_status)."\n".
            "free_CA_mode: ".sprintf ("%u", $free_CA_mode)."\n".
            "descriptors_loop_length: ".sprintf ("%u", $descriptors_loop_length)."\n";
            if ($descriptors_loop_length>0)
               {
               $descripteurshexa .= unpack ('H*',pack('C*',@tspacket[($compteur+12)..($compteur+11+$descriptors_loop_length)]))."-";            
               $descripteurs .= pack('C*',@tspacket[($compteur+12)..($compteur+11+$descriptors_loop_length)])."-";            
               }
            print $event;
            print $descripteurshexa,"\n";
            print $descripteurs,"\n\n";
            $compteur += 12 + $descriptors_loop_length;
            }
         # convert into hexdump readable format
         my $convhex = unpack ('H*',pack ('C*',@tspacket));
         print $convhex,"\n";
         print $result,"\n";
         }
      }
   }

close (FSOURCE);

die ("End of operation\n");

So, what we look for, at that stage is the short_event_descriptor.
According to EN 300468 (free access @ETSI), its tag is 0x4D (see table 12, possible location of descriptors).
Its structure is explained in §6.2.37 table 85:

Código:

short_event_descriptor(){
descriptor_tag 8 uimsbf
descriptor_length 8 uimsbf
ISO_639_language_code 24 bslbf
event_name_length 8 uimsbf
for (i=0;i<event_name_length;i++){
event_name_char 8 uimsbf
}
text_length 8 uimsbf
for (i=0;i<text_length;i++){
text_char 8 uimsbf
}
}

Among all these names that are brought over EIT actual p/f, our goal is to chose the right one, to clean it, and then to rename the file (or create a folder) from it...

Semantics for the short event descriptor:
ISO_639_language_code: This 24-bit field contains the ISO 639-2 [42] three character language code of the language
of the following text fields. Both ISO 639-2/B and ISO 639-2/T may be used. Each character is coded into 8 bits
according to ISO/IEC 8859-1 [23] and inserted in order into the 24-bit field.
EXAMPLE: French has 3-character code "fre", which is coded as:
"0110 0110 0111 0010 0110 0101".
event_name_length: An 8-bit field specifying the length in bytes of the event name.
event_name_char: This is an 8-bit field. A string of "char" fields specifies the event name. Text information is coded
using the character sets and methods described in annex A.
text_length: This 8-bit field specifies the length in bytes of the following text describing the event.
text_char: This is an 8-bit field. A string of "char" fields specify the text description for the event. Text information is
coded using the character sets and methods described in annex A.

other descriptor are of the same structure, ie. TLV, so we can ignore them till we find the right tag, hoping the short_event descriptor is contained into the first TS packet we found (otherwise, it's harder).

wowww
Thanks for the investigations and protos.
Lot's of stuff to read and test!!!

Lot of usefull ideas comes to mind,
for instance,
lg , when idle, can navigate throught all the channels and retrieve the information of programs in every channel,
with its start time and end time.

We can think in the implementation of a programming manager, that is we select the program name: Gormittis, for isntance,
and then lg will automatically saves all the gormittis programs that are in all channels.
In spain for instance, we have gormittis in La 1 , and clantv.

Would this be feasible?

Hi Vic,

In theory this could be feasible (I am awaiting such a box that would be able to program itself to record any programs related to a specific keyword ), but I think there are some important issues to deal with:
- integrating things within the LG is really hard (workaround it the master word since we have no SDK, no API, etc.), but you guys do miracles there
- it seems that the LG does mostly only manage the stream related to a specific tv (technically, service_id) and probably only gets an overview of the whole multiplex (all tv broadcasted over the same frequency channel) only in "EPG" mode ... so we would probably need to scan all tv services within all frequency channel (this can be long)
- you have then 4 types of EIT: EITp/f or EITs, actual or other. p=present (current event), f=following (immediately following event), s=schedule (later event), actual = event on the current multiplex, other = event on an other multiplex. Within the EIT you can find information about the mux you are looking for, the service_id, etc. And you have different types of EIT (table_id), but they have all the same PID, and most of them are spread over multiple TS packets. So one of the first task is to get them all, rebuild the packet, store them, link them to all channels, etc. Not impossible therefore, but huge task, and even huger due do LG lack of support.
- the EIT are not all well set. I don't know for Spain, but in France, we have EITp/f actual which are mandatory, EITp/f other that are meant to be mandatory (but not always transported) and there are no obligation yet about EITs. Therefore we are only sure, by scanning all channels, all services, all EIT, to get information about current and following programs only (and only on national services to tell everything). This should improve, hopefully. Moreover, the way the EIT are edited, today, is disconnected from the way the tv service is build and there are therefore some shifts between the actual timing of program broadcast and the timing told within the EIT. And last but not least there are some errors sometimes in the EIT (we had big bugs in June this year on one of our major TV service that brought down some TV during the WorldCup evenings...), and you therefore need to develop something robust.

So, yes, it could be feasible, if tv services bring enough information over the air, and if you develop many things to bypass what the LG does (or does not). I am afraid this would be a really huge task!

Cheers,
Matt

Ok, here is the perl code for MS windows that extracts (among other things - and that's what I believe ) the name of the events from the first TS packet only of all EITp/f actual it finds... (still so many limitation). It prints it, and other things, in the "event_name" line ... and for the time being it only shows it "raw" i.e. with no localization taken into account (the first char shows which character code page should be taken) ...

Código:


#!/usr/bin/perl
my ($a);
my $cpt = 0;

my $temp = 1 + $#ARGV;
print "Number of parameters: $temp\n\n";

if (!($#ARGV == 0))
   {
   print "Parameters: FSOURCE\n";
   print "FSOURCE: TS file name to search in about EIT actual present/following\n";
   print "In case of specific characters, please use quote around the name of the file\n\n";
   print "This script digs in a TS file, looks for EIT actual present/following\n";
        print "and then print the corresponding TS packet in hexdump form\n\n";
        die ("Nombre d'arguments incorrect");
   }
else {
   $a=$ARGV[0];
   print "provided FSOURCE is: $a\n";
   }

open (FSOURCE,"<$a") or die("open: $!");
binmode FSOURCE;
my $buffer;
my @tspacket;

while (read (FSOURCE, $buffer, 188))
   {
   @tspacket = unpack 'C*',$buffer;
   if (!($tspacket[0]==0x47)) {die("Corrupted file source or endianess issue (please then replace H by h, C by c, etc. in perl script)\n");}
   if (($tspacket[1]==0x40) and ($tspacket[2]==0x12)) # first section of an EIT
      {
      if (($tspacket[3]>=0x10) and ($tspacket[3]<0x20) and ($tspacket[5]==0x4e)) #check whether this is an EIT actual present/following
         {
         # convert into hexdump readable format
         my $convhex = unpack ('H*',pack ('C*',@tspacket));
         print "\n--------- NEW EIT P/F ACTUAL FIRST TS PACKET ---------";
         print "\n",$convhex,"\n";
         
         my $section_syntax_indicator = (0b1000_0000 & $tspacket[6]) / 0b1000_0000;
         my $section_length = (0b0000_1111 & $tspacket[6])*0x100 + $tspacket[7];
         my $service_id = $tspacket[8] * 0x100 + $tspacket[9];
         my $version_number = ($tspacket[10] & 0b0011_1110) / 0b0000_0010;
         my $current_next_indicator = $tspacket[10] & 0b0000_0001;
         my $section_number = $tspacket[11];
         my $last_section_number = $tspacket[12];
         my $transport_stream_id = $tspacket[13] * 0x100 + $tspacket[14];
         my $original_network_id = $tspacket[15] * 0x100 + $tspacket[16];
         my $segment_last_section_number = $tspacket[17];
         my $last_table_id = $tspacket[18];
         my $result = "section_syntax_indicator: " . sprintf ("%1u",$section_syntax_indicator) . "\n" .
         "section_length: " . sprintf ("%4u",$section_length) . "\n" .
         "service_id: 0x" . sprintf ("%04X", $service_id) . "\n".
         "version_number: ".sprintf ("%02u", $version_number)."\n".
         "current_next_indicator: " . sprintf ("%1u",$current_next_indicator)."\n".
         "section_number: 0x". sprintf ("%02X", $section_number) . "\n".
         "last_section_number: 0x". sprintf ("%02X", $last_section_number) . "\n".
         "transport_stream_id: ".sprintf ("%04u", $transport_stream_id)." (0x".sprintf ("%04X", $transport_stream_id).")\n".
         "original_network_id: ".sprintf ("%04u", $original_network_id)." (0x".sprintf ("%04X", $original_network_id).")\n".
         "segment_last_section_number: 0x". sprintf ("%02X", $segment_last_section_number) . "\n".
         "last_table_id: 0x". sprintf ("%02X", $last_table_id)."\n";
         print $result,"\n";
         my $compteur = 19;
         while (($compteur < 188)) # or !($tspacket[$compteur] == 0xFF))
            {
            my $descripteurshexa = "";
            my $descripteurs = "";
            my $event_id = ($tspacket[$compteur] * 0x100 + $tspacket[($compteur+1)]);
            #my $temp = sprintf("%02X",$tspacket[$compteur]).sprintf("%02X",$tspacket[$compteur+1])." ".sprintf("%04X",$event_id)."\n";
            #print $compteur,$temp,"\n";
            #my $temp = sprintf("%02X",$tspacket[19]).sprintf("%02X",$tspacket[20])." ".sprintf("%04X",$event_id)."\n";
            #print $compteur,$temp,"\n";
            my $start_time = ((($tspacket[$compteur+2] * 0x100 + $tspacket[$compteur+3])*0x100 + $tspacket[$compteur+4])*0x100 + $tspacket[$compteur+5])*0x100 + $tspacket[$compteur+6];
            my $start_timeMJD = ($tspacket[$compteur+2] * 0x100 + $tspacket[$compteur+3]);
            my $start_timeBCD = ($tspacket[$compteur+4]*0x100 + $tspacket[$compteur+5])*0x100 + $tspacket[$compteur+6];
            my $duration = ($tspacket[$compteur+7] * 0x100 + $tspacket[$compteur+8]) * 0x100 + $tspacket[$compteur+9];
            my $running_status = (0b1110_0000 & $tspacket[$compteur+10]) / 0b0010_0000;
            my $free_CA_mode = (0b0001_0000 & $tspacket[$compteur+10]) / 0b0001_0000;
            my $descriptors_loop_length = (0b0000_1111 & $tspacket[$compteur+10]) * 0x100 + $tspacket[$compteur+11]; # BIG warning there, it can goes over the current TS packet, be ready to fill with zero ...
            
            my $event = "event_id: 0x". sprintf ("%04X", $event_id)."\n".
            "start_time: 0x". sprintf ("%010X", $start_time)."\n".
            "start_timeMJD: 0x". sprintf ("%04X", $start_timeMJD)."\n".
            "start_timeBCD: ". $tspacket[$compteur+4].":".$tspacket[$compteur+5].":".$tspacket[$compteur+6] ."\n".
            "duration: ". $tspacket[$compteur+7] .":". $tspacket[$compteur+8].":".$tspacket[$compteur+9] ."\n".
            "running_status: ".sprintf ("%u", $running_status)."\n".
            "free_CA_mode: ".sprintf ("%u", $free_CA_mode)."\n".
            "descriptors_loop_length: ".sprintf ("%u", $descriptors_loop_length)."\n";
            if ($descriptors_loop_length>0)
               {
               $descripteurshexa .= unpack ('H*',pack('C*',@tspacket[($compteur+12)..($compteur+11+$descriptors_loop_length)]))."-";            
               $descripteurs .= pack('C*',@tspacket[($compteur+12)..($compteur+11+$descriptors_loop_length)])."-";            
               }
            print $event;
            print $descripteurshexa,"\n";
            print $descripteurs,"\n";
            
            #now, it's time to dig in descriptor till finding the short_event_descriptor (hopefully in the first TS packet of the EIT)
            my $desc_compteur = 0;
            while ($desc_compteur < $descriptors_loop_length)
               {
               $desc_compteur ++;
               my $descriptor_tag = $tspacket[$compteur + 11 + $desc_compteur];
               my $descriptor_length = $tspacket[$compteur + 11 + $desc_compteur + 1];
               if ($descriptor_tag == 0x4D) #we got it! let's hope it is not split over an other packet
                  {
                  my $integrity_check = (($compteur+11+$desc_compteur+1+$descriptor_length)<=188); #if true, we have it all otherwise it is split over an other TS packet :-/
                  if ($integrity_check)
                     {
                     my $ISO_639_language_code = pack ('C*',@tspacket[($compteur+11+$desc_compteur+2)..($compteur+11+$desc_compteur+4)]);
                     my $event_name_length = $tspacket [($compteur+11+$desc_compteur+5)];
                     my $event_cpt = 0;
                     my $event_name = "";
                     while ($event_cpt < $event_name_length)
                        {
                        $event_cpt ++;
                        $event_name .= pack ('C*',$tspacket [($compteur+11+$desc_compteur+5+$event_cpt)]);
                        }
                     $desc_compteur += 5+$event_name_length;
                     my $text_length = $tspacket [($compteur+11+$desc_compteur+1)];
                     my $text_cpt = 0;
                     my $text = "";
                     while ($text_cpt < $text_length)
                        {
                        $text_cpt ++;
                        $text .= pack ('C*',$tspacket [($compteur+11+$desc_compteur+1+$text_cpt)]);
                        }
                     print "event_name: $event_name \n"; #this is the name we want ... now we have to deal with langage specifics...
                     print "text: $text \n\n";
                     }
                  }
               $desc_compteur += $descriptor_length + 1;
               }
            $compteur += 12 + $descriptors_loop_length;
            }
         }
      }
   }

close (FSOURCE);

die ("End of operation\n");


Thanks,
for me is working fine in linux with no change at all,
with this input: http://dl.dropbox.com/u/684543/varios/ms450h/Matthelas/Gormitis.ts
I get this output: http://dl.dropbox.com/u/684543/varios/ms450h/Matthelas/salida.txt

It seems the information is sent like a loop, continue repeating the information

Hi Vic,

Good news
My code is full of flaws in fact, but the loop is normal. The EITp/f is regularly sent so that whenever you step in the stream you can know what you look at rather than needing to wait for the next program to begin. The output should change around when the program changes (i.e. from news to meteo or from film1 to film2).
Among the different flaws of the code, here are some which need to be dealt with, I think:
- I only check the first TS packet of an EITp/f. To do things correctly, we should gather all TS packets of a single EIT, append them together, and then search within the two events that should be described (p and f events), and then check times to know which one is current, and which one is next.
- I did not integrate "locales" so the output is messy, I have a patch on my last version (not yet online) that replace any non basic alphanumeric character (0-9, a-z, A-Z) by a _ (including space, à,é, etc.), which is also not absolutely perfect (there are tables in the 300468, Annex A that could permit to do a better job but this needs a lot more coding)
- I do not check whether the EITp/f is related to the right TS nor to the right tv service (this means lot more code) and I therefore rely on the fact that the LG does it job correctly and only keeps the EITp/f related to the program recorded (seems this is true since you have only one name in your output I am working on a full TS recording here, and I have outputs for all services of this TS => around 6 names ...)
- Local time indicated in EITp/f is absolute (UTC) and has therefore to be corrected by some Offset that can be found in an other Table ... the TOT (this is an other story ...)

Moreover, I only check EITp/f ... in order to get information about future programs, we should keep other EIT as well (EITs, actual for sure, perhaps other as well) but first make sure we don't use only the first TS packet of each of them which means, as a first step, that we should append all related TS packets belonging to the same EIT, whether p/f, s, actual or other (which means again more coding).

Last, we have to consider some algorithm about chosing the right name, for example keeping all names we find then chosing the second one (to avoid Meteo if our record began 5 minutes earlier) of the one that appears the most during the first hour (we need then to put some counter to estimated at what time each packet corresponds) or absolutely (slower), etc.

Still a long way to go, but ... feasible

If I have some time, I'll work on gathering TS packets related to the same EIT...

Cheers,
Matt

PS: I have currently no access to dropbox :-/ will look your result and get the test stream later

Last version so far, it now also calculates time as entered into EIT (warning, these should be corrected by an offset, since they are UTC => we have to catch the TOT table to learn about that Offset ...) and I commented most of the print.

The output is now two lines, for each first TS packet of an EITp/f:
The event_name it founds into the first TS packet of the EITp/f with no special characters (i.e. only 0-9,a-z,A-Z, everything else is replaced by _)
A line about the TS packet where that info was found + with a proposed name according to currently studied EITp/f (i.e. more or less what should do the LG when it decides to name a record, from the first EITp/f it finds), appending event_name with date, UTC time and service_id (sorry, to catch the actual number of the tv service we have to look in an other table ... the NIT ... or in the parameters of the LG if numbering was changed)
That name could be used to rename the file (once we have seen enough of them to chose which one is the best one) ... but that's an other story.

Código:


#!/usr/bin/perl
my ($a);
my $cpt = 0;

my $temp = 1 + $#ARGV;
print "Number of parameters: $temp\n\n";

if (!($#ARGV == 0))
   {
   print "Parameters: FSOURCE\n";
   print "FSOURCE: TS file name to search in about EIT actual present/following\n";
   print "In case of specific characters, please use quote around the name of the file\n\n";
   print "This script digs in a TS file, looks for EIT actual present/following\n";
        print "and then print the corresponding TS packet in hexdump form\n\n";
        die ("Nombre d'arguments incorrect");
   }
else {
   $a=$ARGV[0];
   print "provided FSOURCE is: $a\n";
   }

open (FSOURCE,"<$a") or die("open: $!");
binmode FSOURCE;
my $buffer;
my @tspacket;
my $packet_ref=0;

while (read (FSOURCE, $buffer, 188))
   {
   @tspacket = unpack 'C*',$buffer;
   if (!($tspacket[0]==0x47)) {die("Corrupted file source or endianess issue (please then replace H by h, C by c, etc. in perl script)\n");}
   if (($tspacket[1]==0x40) and ($tspacket[2]==0x12)) # first TS packet of an EIT
      {
      if (($tspacket[3]>=0x10) and ($tspacket[3]<0x20) and ($tspacket[5]==0x4e)) #check whether this is an EIT actual present/following
         {
         # convert into hexdump readable format
         my $convhex = unpack ('H*',pack ('C*',@tspacket));
         #print "\n--------- NEW EIT P/F ACTUAL FIRST TS PACKET ---------";
         #print "\n",$convhex,"\n";
         
         my $section_syntax_indicator = (0b1000_0000 & $tspacket[6]) / 0b1000_0000;
         my $section_length = (0b0000_1111 & $tspacket[6])*0x100 + $tspacket[7];
         my $service_id = $tspacket[8] * 0x100 + $tspacket[9];
         my $version_number = ($tspacket[10] & 0b0011_1110) / 0b0000_0010;
         my $current_next_indicator = $tspacket[10] & 0b0000_0001;
         my $section_number = $tspacket[11];
         my $last_section_number = $tspacket[12];
         my $transport_stream_id = $tspacket[13] * 0x100 + $tspacket[14];
         my $original_network_id = $tspacket[15] * 0x100 + $tspacket[16];
         my $segment_last_section_number = $tspacket[17];
         my $last_table_id = $tspacket[18];
         my $result = "section_syntax_indicator: " . sprintf ("%1u",$section_syntax_indicator) . "\n" .
         "section_length: " . sprintf ("%4u",$section_length) . "\n" .
         "service_id: 0x" . sprintf ("%04X", $service_id) . "\n".
         "version_number: ".sprintf ("%02u", $version_number)."\n".
         "current_next_indicator: " . sprintf ("%1u",$current_next_indicator)."\n".
         "section_number: 0x". sprintf ("%02X", $section_number) . "\n".
         "last_section_number: 0x". sprintf ("%02X", $last_section_number) . "\n".
         "transport_stream_id: ".sprintf ("%04u", $transport_stream_id)." (0x".sprintf ("%04X", $transport_stream_id).")\n".
         "original_network_id: ".sprintf ("%04u", $original_network_id)." (0x".sprintf ("%04X", $original_network_id).")\n".
         "segment_last_section_number: 0x". sprintf ("%02X", $segment_last_section_number) . "\n".
         "last_table_id: 0x". sprintf ("%02X", $last_table_id)."\n";
         #print $result,"\n";
         my $compteur = 19;
         while (($compteur < 188)) # or !($tspacket[$compteur] == 0xFF))
            {
            my $descripteurshexa = "";
            my $descripteurs = "";
            my $event_id = ($tspacket[$compteur] * 0x100 + $tspacket[($compteur+1)]);
            my $start_time = ((($tspacket[$compteur+2] * 0x100 + $tspacket[$compteur+3])*0x100 + $tspacket[$compteur+4])*0x100 + $tspacket[$compteur+5])*0x100 + $tspacket[$compteur+6];
            my $start_timeMJD = ($tspacket[$compteur+2] * 0x100 + $tspacket[$compteur+3]);
            my $start_timeBCD = ($tspacket[$compteur+4]*0x100 + $tspacket[$compteur+5])*0x100 + $tspacket[$compteur+6];
            my $duration = ($tspacket[$compteur+7] * 0x100 + $tspacket[$compteur+8]) * 0x100 + $tspacket[$compteur+9];
            my $running_status = (0b1110_0000 & $tspacket[$compteur+10]) / 0b0010_0000;
            my $free_CA_mode = (0b0001_0000 & $tspacket[$compteur+10]) / 0b0001_0000;
            my $descriptors_loop_length = (0b0000_1111 & $tspacket[$compteur+10]) * 0x100 + $tspacket[$compteur+11]; # BIG warning there, it can goes over the current TS packet, be ready to fill with zero ...
            
            # calculate the MJD time
            my $MJD_Yprime = int ( ($start_timeMJD - 15078.2) / 365.25 ); #printf "Y': $MJD_Yprime\n";
            my $MJD_Mprime = int ( ($start_timeMJD - 14956.1 - int ($MJD_Yprime * 365.25) ) / 30.6001 );  #printf "M': $MJD_Mprime\n";
            my $MJD_Day = $start_timeMJD - 14956 - int ($MJD_Yprime * 365.25) - int ($MJD_Mprime * 30.6001);  #printf "Day': $MJD_Day\n";
            my $MJD_K = 0;
            if (($MJD_Mprime == 14) or ($MJD_Mprime == 15)) {$MJD_K = 1;}
            # printf "K: $MJD_K\n";
            my $MJD_Year = $MJD_Yprime + $MJD_K + 1900;  #printf "Year: $MJD_Year\n";
            my $MJD_Month = $MJD_Mprime - 1 - $MJD_K * 12; #printf "Month: $MJD_Month\n";
            my $MJD_Date = sprintf ("%04u", $MJD_Year)."-".sprintf ("%02u", $MJD_Month)."-".sprintf ("%02u", $MJD_Day);
            my $BCD_Time = sprintf("%02u",$tspacket[$compteur+4])."-".sprintf("%02u",$tspacket[$compteur+5])."-".sprintf("%02u",$tspacket[$compteur+6]);
            
            my $event = "event_id: 0x". sprintf ("%04X", $event_id)."\n".
            "start_time: 0x". sprintf ("%010X", $start_time)."\n".
            "start_timeMJD: 0x". sprintf ("%04X", $start_timeMJD)." ($MJD_Date : YYYY-MM-DD)\n".
            "start_timeBCD: 0x". sprintf ("%06X", $start_timeMJD)." ($BCD_Time : HH-MM-SS)\n".
            "duration: ". $tspacket[$compteur+7] ."-". $tspacket[$compteur+8]."-".$tspacket[$compteur+9] ."\n".
            "running_status: ".sprintf ("%u", $running_status)."\n".
            "free_CA_mode: ".sprintf ("%u", $free_CA_mode)."\n".
            "descriptors_loop_length: ".sprintf ("%u", $descriptors_loop_length)."\n";
            if ($descriptors_loop_length>0)
               {
               $descripteurshexa .= unpack ('H*',pack('C*',@tspacket[($compteur+12)..($compteur+11+$descriptors_loop_length)]))."-";            
               $descripteurs .= pack('C*',@tspacket[($compteur+12)..($compteur+11+$descriptors_loop_length)])."-";            
               }
            #print $event;
            #print $descripteurshexa,"\n";
            #print $descripteurs,"\n";
            
            #now, it's time to dig in descriptor till finding the short_event_descriptor (hopefully in the first TS packet of the EIT)
            my $desc_compteur = 0;
            while ($desc_compteur < $descriptors_loop_length)
               {
               $desc_compteur ++;
               my $descriptor_tag = $tspacket[$compteur + 11 + $desc_compteur];
               my $descriptor_length = $tspacket[$compteur + 11 + $desc_compteur + 1];
               if ($descriptor_tag == 0x4D) #we got it! let's hope it is not split over an other packet
                  {
                  my $integrity_check = (($compteur+11+$desc_compteur+1+$descriptor_length)<=188); #if true, we have it all otherwise it is split over an other TS packet :-/
                  if ($integrity_check)
                     {
                     my $ISO_639_language_code = pack ('C*',@tspacket[($compteur+11+$desc_compteur+2)..($compteur+11+$desc_compteur+4)]);
                     my $event_name_length = $tspacket [($compteur+11+$desc_compteur+5)];
                     my $event_cpt = 0;
                     my $event_name = "";
                     my $event_name_first_char = $tspacket [($compteur+11+$desc_compteur+6)];
                     while ($event_cpt < $event_name_length)
                        {
                        $event_cpt ++;
                        $event_name .= pack ('C*',$tspacket [($compteur+11+$desc_compteur+5+$event_cpt)]);
                        }
                     $desc_compteur += 5+$event_name_length;
                     my $text_length = $tspacket [($compteur+11+$desc_compteur+1)];
                     my $text_cpt = 0;
                     my $text = "";
                     my $text_first_char = $tspacket [($compteur+11+$desc_compteur+2)];
                     while ($text_cpt < $text_length)
                        {
                        $text_cpt ++;
                        $text .= pack ('C*',$tspacket [($compteur+11+$desc_compteur+1+$text_cpt)]);
                        }
                     
                     # warning, following check makes the assumption that text is pre-coded correctly ... this is not always the case
                     if (!(($event_name_first_char <= 0xFF) and ($event_name_first_char >= 0x20))) #check whether we should use a local character code page
                        { #specific character code page
                        # first, we remove the first char
                        my @temp_event_str = split(//,$event_name);
                        shift (@temp_event_str);
                        #now we have to clean each char and replace it with an english one (or underscore) ... can be hard stuff
                        my $temp_event_str_cpt = 0;
                        while ($temp_event_str_cpt<= $#temp_event_str)
                           {
                           my $check_evt = unpack('C*',$temp_event_str[$temp_event_str_cpt]);
                           #my $hex_check_event = sprintf("%02X",unpack('C*',$check_evt));
                           #print "$check_evt $hex_check_event\n";
                           if   (!(
                              (($check_evt >= 0x30) and ($check_evt<=0x39)) or
                              (($check_evt >= 0x41) and ($check_evt<=0x5A)) or
                              (($check_evt >= 0x61) and ($check_evt<=0x7A))
                              ))
                              {
                              $temp_event_str[$temp_event_str_cpt] = '_'; # set a "_"
                              }
                           $temp_event_str_cpt ++;
                           }
                        #now we can recreate the target name
                        $event_name = join('',@temp_event_str);
                        }
                        print "event_name: $event_name \n"; #this is the name we want with _ instead of specific chars, if first char is correct
                        print "proposed_name from partial EIT found in TS packet ($packet_ref): $event_name-$MJD_Date-at-$BCD_Time-on-$service_id\n\n";
                     # if (!(($text_first_char <= 0xFF) and ($text_first_char>=0x20))) #check whether we should use a local character code page
                        # { #specific character code page
                        # print "text: $text \n\n"; #still needs to be cleaned
                        # }
                        # else
                        # { #generic character code page
                        # print "text: $text \n\n";
                        # }
                     }
                  }
               $desc_compteur += $descriptor_length + 1;
               }
            $compteur += 12 + $descriptors_loop_length;
            }
         }
      }
   $packet_ref ++;
   }

close (FSOURCE);

die ("End of operation\n");


Cheers,
Matt

Last investigation about EIT ... now I consider the whole EIT content, not only the first TS packet, and if possible all EIT (p/f, s, actual, other).

I had a look at one multiplex here in France and it seems that the following is applied:
- continuity counter is working for all type of EIT, with no difference => a single counter for all of them (checked on EITp/f actual and other ... I'll check on an other multiplex to see if that's also true for EITs, but I guess it is the case)
- the first TS packet has a big header (the one the perl script is digging into) but the following TS packets have no header except the TS header itself (i.e. only 4 octets) and the data is to be appended directly to the one of the previous TS packet if continuity counter is ok

So, not too hard.
New script coming soon, only to gather TS packets ...

Edit: probably bad news, I am pretty sure that the LG filters out the EIT not related to the current service it records, and probably the one related to future event as well (less sure), therefore from .ts recorded by the LG we will probably only see EITp/f actual related to the recorded tv service, i.e. only current and following events (changing at each programs) but we probably won't be able to look at what is planned in a latter future :-/
There are still things to do for automatic renaming anyway...

Cheers,
Matt