EIT (Event Information Table) packets overview - ALPHA Phase: investigation and proof of concept only!!!

As promised, here is a new script that gathers all TS packets related to a same EIT by checking (and only checking that so far) the continuity_counter ... there is no integrity check, nor any parsing of the EIT packet itself (but that's what was partly done in the previous scripts).

Next step is therefore to mix this script and previous one to make sure we fully parse all EIT we catch, whether actual, other, p/f or s ... Then I'll retreat

Cheers,
Matt

Código:


#!/usr/bin/perl
my ($a);
my $cpt = 0;

my $temp = 1 + $#ARGV;
print "Number of parameters: $temp\n\n";

if (!($#ARGV == 0))
   {
   print "Parameters: FSOURCE\n";
   print "FSOURCE: TS file name to search in about EIT \n";
   print "In case of specific characters, please use quote around the name of the file\n\n";
   print "This script digs in a TS file, looks for EIT \n";
        print "and then concatenate the corresponding TS packets (no integrity check, only continuity check)\n\n";
        die ("Nombre d'arguments incorrect");
   }
else {
   $a=$ARGV[0];
   print "provided FSOURCE is: $a\n";
   }

open (FSOURCE,"<$a") or die("open: $!");
binmode FSOURCE;

my $buffer;
my @tsheader;
my @tspacket;
my $previous_continuity_counter;
my $continuity_check=0;
my $TABLErebuilding=0;
my $EITPackets = 0;
my @EIT;
my $packet_ref=0;

while (read (FSOURCE, $buffer, 188))
   {
   # TS Header => true for all TS packets
   @tsheader = unpack 'C4',$buffer;
   my $sync_byte = $tsheader[0];
   my $transport_error_indicator = ($tsheader[1]&0b1000_0000)/0b1000_0000;
   my $payload_unit_start_indicator = ($tsheader[1]&0b0100_0000)/0b0100_0000;
   my $transport_priority = ($tsheader[1]&0b0010_0000)/0b0010_0000;
   my $PID =  ($tsheader[1]*0x100 + $tsheader[2])&0b0001_1111_1111_1111;
   my $transport_scrambling_control = ($tsheader[3]&0b1100_0000)/0b0100_0000;
   my $adaptation_field_control = ($tsheader[3]&0b0011_0000)/0b0001_0000;
   my $continuity_counter= ($tsheader[3]&0b0000_1111);
   
   # check of sync_byte presence ... if not, there is an issue with the file = quit
   if (!($sync_byte==0x47)) {die("Corrupted file source or endianess issue (please then replace H by h, C by c, etc. in perl script)\n");}
   
   # we only work on EIT => PID 0x12
   if ($PID == 0x12)
      {
      
      # #print each EIT TS packet, whether continuous, good or bad, etc.
      # my $file_offset = sprintf "%X",($packet_ref * 188);
      # print "\nEIT TS packet ($packet_ref - offset: 0x$file_offset) from EIT try $EITPackets continuity_counter is: $continuity_counter\n";
      # print "TS packet is:\n";
      # my $convhex = unpack ('H*',$buffer);
      # print $convhex,"\n";

      if (($payload_unit_start_indicator == 0b1) and ($adaptation_field_control == 0b01)) #we don't want to dig in adaptation field yet
         {# this is a first TS packet of an EIT, no continuity_check needed
         # if there are any previous EIT packet, time to deal with it ...
         if ($EITPackets >0)
            {
            #do whatever you want with previous @EIT
            # parse (@EIT)
            
            # convert into hexdump readable format
            my $asciihex = pack ('C*',@EIT);
            my $convhex = unpack ('H*',$asciihex);
            my $EIT_length = $#EIT + 1;
            my $EIT_num = $EITPackets - 1;
            print "--------- EIT (num:$EIT_num - lenght:$EIT_length)---------\n";
            print $convhex,"\n";
            #print $asciihex,"\n";
            
            # prepare next packet operation
            $#EIT = -1; # we reset the array
            }
         $EITPackets ++;
         $previous_continuity_counter = $continuity_counter; # for next packet checks
         $TABLErebuilding =1;
         @tspacket = unpack 'C*',$buffer;
         shift (@tspacket);
         shift (@tspacket);
         shift (@tspacket);
         shift (@tspacket);
         my $pointer_field = shift (@tspacket);
         if (!($pointer_field == 0x00))
            {
            $TABLErebuilding =0;
            $#EIT = -1;
            }
         else
            {
            my $temp_cpt =0;
            foreach $element (@tspacket)
               {
               $EIT [$temp_cpt] = $element;
               $temp_cpt ++;
               }
            }
         }
      elsif (($TABLErebuilding == 1) and ($adaptation_field_control == 0b01))
         {# this is a continuing TS packet of an EIT and we have a precedent TS packet that could match + no adaptation field
         #check continuity counter
         if ((($previous_continuity_counter + 0b1)&0b0000_1111) == $continuity_counter)
            {# we can keep appending data
            $previous_continuity_counter = $continuity_counter; # for next packet checks
            @tspacket = unpack 'C*',$buffer;
            shift (@tspacket);
            shift (@tspacket);
            shift (@tspacket);
            shift (@tspacket);
            my $EIT_offset = $#EIT + 1;
            my $temp_cpt =0;
            foreach $element (@tspacket)
               {
               $EIT [$EIT_offset + $temp_cpt] = $element;
               $temp_cpt ++;
               }
            }
         else
            {#discontinuity
            $TABLErebuilding =0;
            $#EIT = -1;
            $EITPackets --;
            }
         }
      }
   $packet_ref ++;
   }

close (FSOURCE);

die ("End of operation\n");


Last script. This one concatenates all EIT TS packet it finds, check the TS packet continuity, extract the name of the event and gives access to some other values (UTC times, etc.)

Still for investigation only.

Cheers,
Matt

Código:


#!/usr/bin/perl
use strict;
use warnings;
my ($a);

my $temp = 1 + $#ARGV;
print "Number of parameters: $temp\n\n";

if (!($#ARGV == 0))
   {
   print "Parameters: FSOURCE\n";
   print "FSOURCE: TS file name to search in about EIT \n";
   print "In case of specific characters, please use quote around the name of the file\n\n";
   print "This script digs in a TS file, looks for EIT \n";
    print "and then concatenate the corresponding TS packets before parsing the EIT\n\n";
    die ("Nombre d'arguments incorrect");
   }
else {
   $a=$ARGV[0];
   print "provided FSOURCE is: $a\n";
   }

open (FSOURCE,"<$a") or die("open: $!");
binmode FSOURCE;

my $buffer;
my @tsheader;
my @tspacket;
my $previous_continuity_counter;
my $TABLErebuilding=0;
my $EITPackets = 0;
my @EIT;
my $packet_ref=0;

sub PrintHexaArray
   {
   my $convhex = unpack('H*',pack('C*',@_));
   print $convhex,"\n";
   }

sub PrintHexaBuffer
   {
   my $convhex = unpack('H*',$_[0]);
   print $convhex,"\n";
   }
   
while (read (FSOURCE, $buffer, 188))
   {
   # TS Header => true for all TS packets
   @tsheader = unpack 'C4',$buffer;
   my $sync_byte = $tsheader[0];
   my $transport_error_indicator = ($tsheader[1]&0b1000_0000)/0b1000_0000;
   my $payload_unit_start_indicator = ($tsheader[1]&0b0100_0000)/0b0100_0000;
   my $transport_priority = ($tsheader[1]&0b0010_0000)/0b0010_0000;
   my $PID =  ($tsheader[1]*0x100 + $tsheader[2])&0b0001_1111_1111_1111;
   my $transport_scrambling_control = ($tsheader[3]&0b1100_0000)/0b0100_0000;
   my $adaptation_field_control = ($tsheader[3]&0b0011_0000)/0b0001_0000;
   my $continuity_counter= ($tsheader[3]&0b0000_1111);
   # print "Current packet header: 0x";
   # &PrintHexaArray(@tsheader);
   
   # check of sync_byte presence ... if not, there is an issue with the file = quit
   if (!($sync_byte==0x47)) {die("Corrupted file source or endianess issue (please then replace H by h, C by c, etc. in perl script)\n");}
   
   # we only work on EIT => PID 0x12
   if ($PID == 0x12)
      {
      if (($payload_unit_start_indicator == 0b1) and ($adaptation_field_control == 0b01)) #we don't want to dig in adaptation field yet
         {# this is a first TS packet of an EIT, no continuity_check needed
         
         # if there are any previous EIT packet, time to deal with it ...
         if ($EITPackets >0)
            {
            #############################################
            # do whatever needed with current EIT packet
            #############################################
            print "EIT try $EITPackets gives:\n";
            print "\$#EIT: ",$#EIT," (0x",sprintf ("%X",$#EIT),")\n";
            print "Length from EIT[1]&[2]:",sprintf ("%u",($EIT[1]*0x100+$EIT[2])&0b0000_1111_1111_1111)," (0x",sprintf ("%04X",($EIT[1]*0x100+$EIT[2])&0b0000_1111_1111_1111),")\n";
            &PrintHexaArray(@EIT);
            print "Without padding 0xFF octets:\n";
            $#EIT = (($EIT[1]*0x100+$EIT[2])&0b0000_1111_1111_1111) + 3 - 1;
            &PrintHexaArray(@EIT);
            print "-------------------------------------------------\n\n";
            
            #############################################
            # Let's try the parsing of array @EIT ...
            my $table_id = $EIT[0];
            my $section_syntax_indicator = ($EIT[1]&0b1000_0000)/0b1000_0000;
            my $section_length = ($EIT[1]*0x100 + $EIT[2]) & 0b0000_1111_1111_1111;
            my $service_id =  ($EIT[3]*0x100 + $EIT[4]);
            my $version_number = ($EIT[5]&0b0011_1110) / 0b0000_0010;
            my $current_next_indicator = ($EIT[5]&0b0000_0001);
            my $section_number = $EIT[6];
            my $last_section_number = $EIT[7];
            my $transport_stream_id = ($EIT[8]*0x100 + $EIT[9]);
            my $original_network_id = ($EIT[10]*0x100 + $EIT[11]);
            my $segment_last_section_number = $EIT[12];
            my $last_table_id = $EIT[13];
            my $EIT_cpt = 13;
            
            while (($EIT_cpt < ($section_length + 3 - 11)) and ($EIT_cpt<= $#EIT))
               {
               $EIT_cpt ++;
               my $event_id = $EIT[$EIT_cpt]*0x100 + $EIT[$EIT_cpt+1];
               my $start_time = ((($EIT[$EIT_cpt+2] * 0x100 + $EIT[$EIT_cpt+3])*0x100 + $EIT[$EIT_cpt+4])*0x100 + $EIT[$EIT_cpt+5])*0x100 + $EIT[$EIT_cpt+6];
               my $start_timeMJD = ($EIT[$EIT_cpt+2] * 0x100 + $EIT[$EIT_cpt+3]);
               my $start_timeBCD = ($EIT[$EIT_cpt+4]*0x100 + $EIT[$EIT_cpt+5])*0x100 + $EIT[$EIT_cpt+6];
               my $duration = ($EIT[$EIT_cpt+7] * 0x100 + $EIT[$EIT_cpt+8]) * 0x100 + $EIT[$EIT_cpt+9];
               my $running_status = (0b1110_0000 & $EIT[$EIT_cpt+10]) / 0b0010_0000;
               my $free_CA_mode = (0b0001_0000 & $EIT[$EIT_cpt+10]) / 0b0001_0000;
               my $descriptors_loop_length = (0b0000_1111 & $EIT[$EIT_cpt+10]) * 0x100 + $EIT[$EIT_cpt+11];

               # calculate the MJD time
               my $MJD_Yprime = int ( ($start_timeMJD - 15078.2) / 365.25 ); #printf "Y': $MJD_Yprime\n";
               my $MJD_Mprime = int ( ($start_timeMJD - 14956.1 - int ($MJD_Yprime * 365.25) ) / 30.6001 );  #printf "M': $MJD_Mprime\n";
               my $MJD_Day = $start_timeMJD - 14956 - int ($MJD_Yprime * 365.25) - int ($MJD_Mprime * 30.6001);  #printf "Day': $MJD_Day\n";
               my $MJD_K = 0;
               if (($MJD_Mprime == 14) or ($MJD_Mprime == 15)) {$MJD_K = 1;}
               # printf "K: $MJD_K\n";
               my $MJD_Year = $MJD_Yprime + $MJD_K + 1900;  #printf "Year: $MJD_Year\n";
               my $MJD_Month = $MJD_Mprime - 1 - $MJD_K * 12; #printf "Month: $MJD_Month\n";
               my $MJD_Date = sprintf ("%04u", $MJD_Year)."-".sprintf ("%02u", $MJD_Month)."-".sprintf ("%02u", $MJD_Day);
               my $BCD_Time = sprintf("%02u",$EIT[$EIT_cpt+4])."-".sprintf("%02u",$EIT[$EIT_cpt+5])."-".sprintf("%02u",$EIT[$EIT_cpt+6]);
               
               my $EIT_desc_cpt = $EIT_cpt+11;
               
               while ($EIT_desc_cpt < ($EIT_cpt+11+$descriptors_loop_length-1))
                  {
                  $EIT_desc_cpt ++;
                  my $descriptor_tag = $EIT[$EIT_desc_cpt];
                  my $descriptor_length = $EIT[$EIT_desc_cpt+1];
                  my @descriptor_data; #[0..($descriptor_length-1)];
                  my $desc_element; my $desc_cpt =0;
                  foreach $desc_element (@EIT[($EIT_desc_cpt+2)..($EIT_desc_cpt+1+$descriptor_length)])
                     {
                     $descriptor_data[$desc_cpt]=$desc_element;
                     $desc_cpt ++;
                     }
                  ######################################
                  # DO THINGS HERE with descriptor_data
                  #
                  if (($descriptor_tag == 0x4D) and (($EIT_desc_cpt+1+$descriptor_length)<$section_length + 3)) #we got it whole
                     {
                     my $ISO_639_language_code = pack ('C3',shift (@descriptor_data),shift (@descriptor_data),shift (@descriptor_data));
                     my $event_name_length = shift (@descriptor_data);
                     my $event_cpt = 0;
                     my $event_name = "";
                     my $event_name_first_char = $descriptor_data [0];
                     while ($event_cpt < $event_name_length)
                        {
                        $event_cpt ++;
                        $event_name .= pack ('C*',shift (@descriptor_data));
                        }
                     
                     my $text_length = shift (@descriptor_data);
                     my $text_cpt = 0;
                     my $text = "";
                     my $text_first_char = $descriptor_data[0];
                     while ($text_cpt < $text_length)
                        {
                        $text_cpt ++;
                        $text .= pack ('C*',shift (@descriptor_data));
                        }
                     
                     # warning, following check makes the assumption that text is pre-coded correctly ... this is not always the case
                     if (!(($event_name_first_char <= 0xFF) and ($event_name_first_char >= 0x20))) #check whether we should use a local character code page
                        { #specific character code page
                        # first, we remove the first char
                        my @temp_event_str = split(//,$event_name);
                        shift (@temp_event_str);
                        #now we have to clean each char and replace it with an english one (or underscore) ... can be hard stuff
                        my $temp_event_str_cpt = 0;
                        while ($temp_event_str_cpt<= $#temp_event_str)
                           {
                           my $check_evt = unpack('C*',$temp_event_str[$temp_event_str_cpt]);
                           #my $hex_check_event = sprintf("%02X",unpack('C*',$check_evt));
                           #print "$check_evt $hex_check_event\n";
                           if   (!(
                              (($check_evt >= 0x30) and ($check_evt<=0x39)) or
                              (($check_evt >= 0x41) and ($check_evt<=0x5A)) or
                              (($check_evt >= 0x61) and ($check_evt<=0x7A))
                              ))
                              {
                              $temp_event_str[$temp_event_str_cpt] = '_'; # set a "_"
                              }
                           $temp_event_str_cpt ++;
                           }
                        #now we can recreate the target name
                        $event_name = join('',@temp_event_str);
                        }
                        print "event_name: $event_name \n"; #this is the name we want with _ instead of specific chars, if first char is correct
                        print "proposed_name from EIT $EITPackets: $event_name-$MJD_Date-at-$BCD_Time-on-$service_id\n\n";
                     # if (!(($text_first_char <= 0xFF) and ($text_first_char>=0x20))) #check whether we should use a local character code page
                        # { #specific character code page
                        # print "text: $text \n\n"; #still needs to be cleaned
                        # }
                        # else
                        # { #generic character code page
                        # print "text: $text \n\n";
                        # }
                        
                     }
                  #
                  # END OF DOING THINGS :)
                  ######################################
                  $EIT_desc_cpt += 1+$descriptor_length;
                  }
               $EIT_cpt += 11+$descriptors_loop_length;
               }
            
            #############################################
            # prepare next packet operation
            #############################################
            $#EIT = -1; # we reset the array
            }
         
         $EITPackets ++;
         $previous_continuity_counter = $continuity_counter; # for next packet checks
         $TABLErebuilding =1;
         @tspacket = unpack 'C*',$buffer;
         shift (@tspacket);
         shift (@tspacket);
         shift (@tspacket);
         shift (@tspacket);
         my $pointer_field = shift (@tspacket);
         if (!($pointer_field == 0x00))
            {
            die ();
            $TABLErebuilding =0;
            $#EIT = -1;
            }
         else
            {
            my $temp_cpt =0;
            my $element;
            foreach $element (@tspacket)
               {
               $EIT [$temp_cpt] = $element;
               $temp_cpt ++;
               }
            }
         #############################################
         #print each EIT TS packet, whether continuous, good or bad, etc.
         my $file_offset = sprintf "%X",($packet_ref * 188);
         print "EIT TS 1st packet ($packet_ref - offset: 0x$file_offset) from try $EITPackets - continuity: $continuity_counter\n";
         print "TS packet is:\n";
         &PrintHexaBuffer($buffer);
         }
      elsif (($TABLErebuilding == 1) and ($adaptation_field_control == 0b01))
         {# this is a continuing TS packet of an EIT and we have a precedent TS packet that could match + no adaptation field
         #check continuity counter
         if ((($previous_continuity_counter + 0b1)&0b0000_1111) == $continuity_counter)
            {# we can keep appending data
            $previous_continuity_counter = $continuity_counter; # for next packet checks
            @tspacket = unpack 'C*',$buffer;
            shift (@tspacket);
            shift (@tspacket);
            shift (@tspacket);
            shift (@tspacket);
            my $EIT_offset = $#EIT + 1;
            my $temp_cpt =0;
            my $element;
            foreach $element (@tspacket)
               {
               $EIT [$EIT_offset + $temp_cpt] = $element;
               $temp_cpt ++;
               }
            }
         else
            {#discontinuity
            $TABLErebuilding =0;
            $#EIT = -1;
            $EITPackets --;
            }
         #############################################
         #print each EIT TS packet, whether continuous, good or bad, etc.
         my $file_offset = sprintf "%X",($packet_ref * 188);
         print "EIT TS packet ($packet_ref - offset: 0x$file_offset) from try $EITPackets - continuity: $continuity_counter\n";
         print "TS packet is:\n";
         &PrintHexaBuffer($buffer);
         }
      }
   $#tsheader = -1;
   $#tspacket = -1;
   $packet_ref ++;
   }

close (FSOURCE);

die ("End of operation\n");


Wow !

I leave few days for holidays and when I'm back, you have wrote a script !

Good Work !

++

Seb@stien

Ok, end of proof of concept (I'd say), here is a very last version of the perl script that clean the text of the EIT, with some char correspondance I have chosen to match ISO 8859-15 characters.
I think that this is the table (more or less) used in Spain and in France. I am not sure whether this is the case for Czech as well. In case this is an other one, I can work on it as well, but since this is only a "study" script ...
I also chose to replace space and non breaking space by a _, like most of the special characters, so that if you use the cleaned name to rename a file, it should be ok on most systems.
There are still a lot to do to chose the right name, etc. but that's an other story, I believe, and perhaps a perl script is no more the right way to go then.

Cheers,
Matt

Código:

#!/usr/bin/perl
use strict;
use warnings;
my ($a);

my $temp = 1 + $#ARGV;
print "Number of parameters: $temp\n\n";

if (!($#ARGV == 0))
   {
   print "Parameters: FSOURCE\n";
   print "FSOURCE: TS file name to search in about EIT \n";
   print "In case of specific characters, please use quote around the name of the file\n\n";
   print "This script digs in a TS file, looks for EIT \n";
    print "and then concatenate the corresponding TS packets before parsing the EIT\n\n";
    die ("Nombre d'arguments incorrect");
   }
else {
   $a=$ARGV[0];
   print "provided FSOURCE is: $a\n";
   }

open (FSOURCE,"<$a") or die("open: $!");
binmode FSOURCE;

my $buffer;
my @tsheader;
my @tspacket;
my $previous_continuity_counter;
my $TABLErebuilding=0;
my $EITPackets = 0;
my @EIT;
my $packet_ref=0;

sub HexToLetter# (single $hexvalue) adpated ISO8859_15 matchin - return a string
   {
   my $hexvalue = $_[0];
   my $char ='_';
   
   # 0x20 is space ... could be left as is, but is replaced by _ here
   # 0xA0 is non breakable space (nbsp) ... could be left as is or replaced by space, but is also replaced by _ here
   
   if ($hexvalue == 0x00) {$char ='';}      elsif ($hexvalue == 0x02) {$char ='';}      elsif ($hexvalue == 0x04) {$char ='';}      elsif ($hexvalue == 0x06) {$char ='';}      elsif ($hexvalue == 0x08) {$char ='';}      elsif ($hexvalue == 0x0A) {$char ='';}      elsif ($hexvalue == 0x0C) {$char ='';}      elsif ($hexvalue == 0x0E) {$char ='';}
   elsif ($hexvalue == 0x10) {$char ='';}      elsif ($hexvalue == 0x12) {$char ='';}      elsif ($hexvalue == 0x14) {$char ='';}      elsif ($hexvalue == 0x16) {$char ='';}      elsif ($hexvalue == 0x18) {$char ='';}      elsif ($hexvalue == 0x1A) {$char ='';}      elsif ($hexvalue == 0x1C) {$char ='';}      elsif ($hexvalue == 0x1E) {$char ='';}
   elsif ($hexvalue == 0x20) {$char ='_';}      elsif ($hexvalue == 0x22) {$char ='_';}      elsif ($hexvalue == 0x24) {$char ='_';}      elsif ($hexvalue == 0x26) {$char ='_';}      elsif ($hexvalue == 0x28) {$char ='_';}      elsif ($hexvalue == 0x2A) {$char ='_';}      elsif ($hexvalue == 0x2C) {$char ='_';}      elsif ($hexvalue == 0x2E) {$char ='.';}
   elsif ($hexvalue == 0x30) {$char ='0';}      elsif ($hexvalue == 0x32) {$char ='2';}      elsif ($hexvalue == 0x34) {$char ='4';}      elsif ($hexvalue == 0x36) {$char ='6';}      elsif ($hexvalue == 0x38) {$char ='8';}      elsif ($hexvalue == 0x3A) {$char ='_';}      elsif ($hexvalue == 0x3C) {$char ='_';}      elsif ($hexvalue == 0x3E) {$char ='_';}
   elsif ($hexvalue == 0x40) {$char ='at';}      elsif ($hexvalue == 0x42) {$char ='B';}      elsif ($hexvalue == 0x44) {$char ='D';}      elsif ($hexvalue == 0x46) {$char ='F';}      elsif ($hexvalue == 0x48) {$char ='H';}      elsif ($hexvalue == 0x4A) {$char ='J';}      elsif ($hexvalue == 0x4C) {$char ='L';}      elsif ($hexvalue == 0x4E) {$char ='N';}
   elsif ($hexvalue == 0x50) {$char ='P';}      elsif ($hexvalue == 0x52) {$char ='R';}      elsif ($hexvalue == 0x54) {$char ='T';}      elsif ($hexvalue == 0x56) {$char ='V';}      elsif ($hexvalue == 0x58) {$char ='X';}      elsif ($hexvalue == 0x5A) {$char ='Z';}      elsif ($hexvalue == 0x5C) {$char ='_';}      elsif ($hexvalue == 0x5E) {$char ='_';}
   elsif ($hexvalue == 0x60) {$char ='_';}      elsif ($hexvalue == 0x62) {$char ='b';}      elsif ($hexvalue == 0x64) {$char ='d';}      elsif ($hexvalue == 0x66) {$char ='f';}      elsif ($hexvalue == 0x68) {$char ='h';}      elsif ($hexvalue == 0x6A) {$char ='j';}      elsif ($hexvalue == 0x6C) {$char ='l';}      elsif ($hexvalue == 0x6E) {$char ='n';}
   elsif ($hexvalue == 0x70) {$char ='p';}      elsif ($hexvalue == 0x72) {$char ='r';}      elsif ($hexvalue == 0x74) {$char ='t';}      elsif ($hexvalue == 0x76) {$char ='v';}      elsif ($hexvalue == 0x78) {$char ='x';}      elsif ($hexvalue == 0x7A) {$char ='z';}      elsif ($hexvalue == 0x7C) {$char ='_';}      elsif ($hexvalue == 0x7E) {$char ='_';}
   elsif ($hexvalue == 0x80) {$char ='';}      elsif ($hexvalue == 0x82) {$char ='';}      elsif ($hexvalue == 0x84) {$char ='';}      elsif ($hexvalue == 0x86) {$char ='';}      elsif ($hexvalue == 0x88) {$char ='';}      elsif ($hexvalue == 0x8A) {$char ='';}      elsif ($hexvalue == 0x8C) {$char ='';}      elsif ($hexvalue == 0x8E) {$char ='';}
   elsif ($hexvalue == 0x90) {$char ='';}      elsif ($hexvalue == 0x92) {$char ='';}      elsif ($hexvalue == 0x94) {$char ='';}      elsif ($hexvalue == 0x96) {$char ='';}      elsif ($hexvalue == 0x98) {$char ='';}      elsif ($hexvalue == 0x9A) {$char ='';}      elsif ($hexvalue == 0x9C) {$char ='';}      elsif ($hexvalue == 0x9E) {$char ='';}
   elsif ($hexvalue == 0xA0) {$char ='_';}      elsif ($hexvalue == 0xA2) {$char ='_';}      elsif ($hexvalue == 0xA4) {$char ='_';}      elsif ($hexvalue == 0xA6) {$char ='S';}      elsif ($hexvalue == 0xA8) {$char ='s';}      elsif ($hexvalue == 0xAA) {$char ='_';}      elsif ($hexvalue == 0xAC) {$char ='_';}      elsif ($hexvalue == 0xAE) {$char ='_';}
   elsif ($hexvalue == 0xB0) {$char ='_';}      elsif ($hexvalue == 0xB2) {$char ='_';}      elsif ($hexvalue == 0xB4) {$char ='Z';}      elsif ($hexvalue == 0xB6) {$char ='_';}      elsif ($hexvalue == 0xB8) {$char ='z';}      elsif ($hexvalue == 0xBA) {$char ='_';}      elsif ($hexvalue == 0xBC) {$char ='OE';}      elsif ($hexvalue == 0xBE) {$char ='Y';}
   elsif ($hexvalue == 0xC0) {$char ='A';}      elsif ($hexvalue == 0xC2) {$char ='A';}      elsif ($hexvalue == 0xC4) {$char ='A';}      elsif ($hexvalue == 0xC6) {$char ='AE';}      elsif ($hexvalue == 0xC8) {$char ='E';}      elsif ($hexvalue == 0xCA) {$char ='E';}      elsif ($hexvalue == 0xCC) {$char ='I';}      elsif ($hexvalue == 0xCE) {$char ='I';}
   elsif ($hexvalue == 0xD0) {$char ='D';}      elsif ($hexvalue == 0xD2) {$char ='O';}      elsif ($hexvalue == 0xD4) {$char ='O';}      elsif ($hexvalue == 0xD6) {$char ='O';}      elsif ($hexvalue == 0xD8) {$char ='0';}      elsif ($hexvalue == 0xDA) {$char ='U';}      elsif ($hexvalue == 0xDC) {$char ='U';}      elsif ($hexvalue == 0xDE) {$char ='_';}
   elsif ($hexvalue == 0xE0) {$char ='a';}      elsif ($hexvalue == 0xE2) {$char ='a';}      elsif ($hexvalue == 0xE4) {$char ='a';}      elsif ($hexvalue == 0xE6) {$char ='a';}      elsif ($hexvalue == 0xE8) {$char ='e';}      elsif ($hexvalue == 0xEA) {$char ='e';}      elsif ($hexvalue == 0xEC) {$char ='i';}      elsif ($hexvalue == 0xEE) {$char ='i';}
   elsif ($hexvalue == 0xF0) {$char ='o';}      elsif ($hexvalue == 0xF2) {$char ='o';}      elsif ($hexvalue == 0xF4) {$char ='o';}      elsif ($hexvalue == 0xF6) {$char ='o';}      elsif ($hexvalue == 0xF8) {$char ='o';}      elsif ($hexvalue == 0xFA) {$char ='u';}      elsif ($hexvalue == 0xFC) {$char ='u';}      elsif ($hexvalue == 0xFE) {$char ='';}
   elsif ($hexvalue == 0x01) {$char ='';}      elsif ($hexvalue == 0x03) {$char ='';}      elsif ($hexvalue == 0x05) {$char ='';}      elsif ($hexvalue == 0x07) {$char ='';}      elsif ($hexvalue == 0x09) {$char ='';}      elsif ($hexvalue == 0x0B) {$char ='';}      elsif ($hexvalue == 0x0D) {$char ='';}      elsif ($hexvalue == 0x0F) {$char ='';}
   elsif ($hexvalue == 0x11) {$char ='';}      elsif ($hexvalue == 0x13) {$char ='';}      elsif ($hexvalue == 0x15) {$char ='';}      elsif ($hexvalue == 0x17) {$char ='';}      elsif ($hexvalue == 0x19) {$char ='';}      elsif ($hexvalue == 0x1B) {$char ='';}      elsif ($hexvalue == 0x1D) {$char ='';}      elsif ($hexvalue == 0x1F) {$char ='';}
   elsif ($hexvalue == 0x21) {$char ='_';}      elsif ($hexvalue == 0x23) {$char ='_';}      elsif ($hexvalue == 0x25) {$char ='_';}      elsif ($hexvalue == 0x27) {$char ='_';}      elsif ($hexvalue == 0x29) {$char ='_';}      elsif ($hexvalue == 0x2B) {$char ='_';}      elsif ($hexvalue == 0x2D) {$char ='-';}      elsif ($hexvalue == 0x2F) {$char ='_';}
   elsif ($hexvalue == 0x31) {$char ='1';}      elsif ($hexvalue == 0x33) {$char ='3';}      elsif ($hexvalue == 0x35) {$char ='5';}      elsif ($hexvalue == 0x37) {$char ='7';}      elsif ($hexvalue == 0x39) {$char ='9';}      elsif ($hexvalue == 0x3B) {$char ='_';}      elsif ($hexvalue == 0x3D) {$char ='_';}      elsif ($hexvalue == 0x3F) {$char ='_';}
   elsif ($hexvalue == 0x41) {$char ='A';}      elsif ($hexvalue == 0x43) {$char ='C';}      elsif ($hexvalue == 0x45) {$char ='E';}      elsif ($hexvalue == 0x47) {$char ='G';}      elsif ($hexvalue == 0x49) {$char ='I';}      elsif ($hexvalue == 0x4B) {$char ='K';}      elsif ($hexvalue == 0x4D) {$char ='M';}      elsif ($hexvalue == 0x4F) {$char ='O';}
   elsif ($hexvalue == 0x51) {$char ='Q';}      elsif ($hexvalue == 0x53) {$char ='S';}      elsif ($hexvalue == 0x55) {$char ='U';}      elsif ($hexvalue == 0x57) {$char ='W';}      elsif ($hexvalue == 0x59) {$char ='Y';}      elsif ($hexvalue == 0x5B) {$char ='_';}      elsif ($hexvalue == 0x5D) {$char ='_';}      elsif ($hexvalue == 0x5F) {$char ='_';}
   elsif ($hexvalue == 0x61) {$char ='a';}      elsif ($hexvalue == 0x63) {$char ='c';}      elsif ($hexvalue == 0x65) {$char ='e';}      elsif ($hexvalue == 0x67) {$char ='g';}      elsif ($hexvalue == 0x69) {$char ='i';}      elsif ($hexvalue == 0x6B) {$char ='k';}      elsif ($hexvalue == 0x6D) {$char ='m';}      elsif ($hexvalue == 0x6F) {$char ='o';}
   elsif ($hexvalue == 0x71) {$char ='q';}      elsif ($hexvalue == 0x73) {$char ='s';}      elsif ($hexvalue == 0x75) {$char ='u';}      elsif ($hexvalue == 0x77) {$char ='w';}      elsif ($hexvalue == 0x79) {$char ='y';}      elsif ($hexvalue == 0x7B) {$char ='_';}      elsif ($hexvalue == 0x7D) {$char ='_';}      elsif ($hexvalue == 0x7F) {$char ='';}
   elsif ($hexvalue == 0x81) {$char ='';}      elsif ($hexvalue == 0x83) {$char ='';}      elsif ($hexvalue == 0x85) {$char ='';}      elsif ($hexvalue == 0x87) {$char ='';}      elsif ($hexvalue == 0x89) {$char ='';}      elsif ($hexvalue == 0x8B) {$char ='';}      elsif ($hexvalue == 0x8D) {$char ='';}      elsif ($hexvalue == 0x8F) {$char ='';}
   elsif ($hexvalue == 0x91) {$char ='';}      elsif ($hexvalue == 0x93) {$char ='';}      elsif ($hexvalue == 0x95) {$char ='';}      elsif ($hexvalue == 0x97) {$char ='';}      elsif ($hexvalue == 0x99) {$char ='';}      elsif ($hexvalue == 0x9B) {$char ='';}      elsif ($hexvalue == 0x9D) {$char ='';}      elsif ($hexvalue == 0x9F) {$char ='';}
   elsif ($hexvalue == 0xA1) {$char ='_';}      elsif ($hexvalue == 0xA3) {$char ='_';}      elsif ($hexvalue == 0xA5) {$char ='_';}      elsif ($hexvalue == 0xA7) {$char ='_';}      elsif ($hexvalue == 0xA9) {$char ='_';}      elsif ($hexvalue == 0xAB) {$char ='_';}      elsif ($hexvalue == 0xAD) {$char ='_';}      elsif ($hexvalue == 0xAF) {$char ='_';}
   elsif ($hexvalue == 0xB1) {$char ='_';}      elsif ($hexvalue == 0xB3) {$char ='_';}      elsif ($hexvalue == 0xB5) {$char ='mu';}      elsif ($hexvalue == 0xB7) {$char ='.';}      elsif ($hexvalue == 0xB9) {$char ='_';}      elsif ($hexvalue == 0xBB) {$char ='_';}      elsif ($hexvalue == 0xBD) {$char ='oe';}      elsif ($hexvalue == 0xBF) {$char ='_';}
   elsif ($hexvalue == 0xC1) {$char ='A';}      elsif ($hexvalue == 0xC3) {$char ='A';}      elsif ($hexvalue == 0xC5) {$char ='A';}      elsif ($hexvalue == 0xC7) {$char ='C';}      elsif ($hexvalue == 0xC9) {$char ='E';}      elsif ($hexvalue == 0xCB) {$char ='E';}      elsif ($hexvalue == 0xCD) {$char ='I';}      elsif ($hexvalue == 0xCF) {$char ='I';}
   elsif ($hexvalue == 0xD1) {$char ='N';}      elsif ($hexvalue == 0xD3) {$char ='O';}      elsif ($hexvalue == 0xD5) {$char ='O';}      elsif ($hexvalue == 0xD7) {$char ='_';}      elsif ($hexvalue == 0xD9) {$char ='U';}      elsif ($hexvalue == 0xDB) {$char ='U';}      elsif ($hexvalue == 0xDD) {$char ='Y';}      elsif ($hexvalue == 0xDF) {$char ='ss';}
   elsif ($hexvalue == 0xE1) {$char ='a';}      elsif ($hexvalue == 0xE3) {$char ='a';}      elsif ($hexvalue == 0xE5) {$char ='a';}      elsif ($hexvalue == 0xE7) {$char ='c';}      elsif ($hexvalue == 0xE9) {$char ='e';}      elsif ($hexvalue == 0xEB) {$char ='e';}      elsif ($hexvalue == 0xED) {$char ='i';}      elsif ($hexvalue == 0xEF) {$char ='i';}
   elsif ($hexvalue == 0xF1) {$char ='n';}      elsif ($hexvalue == 0xF3) {$char ='o';}      elsif ($hexvalue == 0xF5) {$char ='o';}      elsif ($hexvalue == 0xF7) {$char ='_';}      elsif ($hexvalue == 0xF9) {$char ='u';}      elsif ($hexvalue == 0xFB) {$char ='u';}      elsif ($hexvalue == 0xFD) {$char ='y';}      elsif ($hexvalue == 0xFF) {$char ='y';}

   return $char;   
   }
   
sub SimpleText # ($string)
   {
   my $string = $_[0];
   my @temp_string = split(//,$string);
   
   my $temp_string_cpt = 0;

   while ($temp_string_cpt<= $#temp_string)
      {
      my $check_evt = unpack('C*',$temp_string[$temp_string_cpt]);
      $temp_string[$temp_string_cpt] = &HexToLetter($check_evt);
      $temp_string_cpt ++;
      }
   #now we can recreate the target name
   $string = join('',@temp_string);
   
   return $string;
   }

sub NameOfSid # ($service_id) returns the name of the channel, according to a static allocation :-/ (could be automated with NIT + SDT)
   { #valid for French DTT only, as of 2nd november 2010 - extracted from www.csa.fr "profil de signalisation" tables
   my $service_id = $_[0];
   my $channel_name = "UnknownTV";
   
   if ($service_id==0x0101) {$channel_name="France 2";}
   elsif ($service_id==0x0104) {$channel_name="France 5";}
   elsif ($service_id==0x0105) {$channel_name="France Ô nationale";}
   elsif ($service_id==0x0106) {$channel_name="LCP";}
   elsif ($service_id==0x0111) {$channel_name="France 3 num 3 a";}
   elsif ($service_id==0x0112) {$channel_name="France 3 num 3 b";}
   elsif ($service_id==0x0113) {$channel_name="France 3 num 3 c";}
   elsif ($service_id==0x0114) {$channel_name="France 3";}
   elsif ($service_id==0x0115) {$channel_name="France 3";}
   elsif ($service_id==0x0116) {$channel_name="France 3";}
   elsif ($service_id==0x0117) {$channel_name="France 3";}
   elsif ($service_id==0x0118) {$channel_name="Fr3 Poitiers";}
   elsif ($service_id==0x0119) {$channel_name="France 3";}
   elsif ($service_id==0x011A) {$channel_name="Fr3 Toulouse";}
   elsif ($service_id==0x011B) {$channel_name="Fr3 Rhône";}
   elsif ($service_id==0x011C) {$channel_name="France 3";}
   elsif ($service_id==0x011D) {$channel_name="France 3";}
   elsif ($service_id==0x011E) {$channel_name="Fr3 Berry";}
   elsif ($service_id==0x011F) {$channel_name="France 3";}
   elsif ($service_id==0x0120) {$channel_name="France 3";}
   elsif ($service_id==0x0121) {$channel_name="France 3";}
   elsif ($service_id==0x0122) {$channel_name="Fr3 LeHavre";}
   elsif ($service_id==0x0123) {$channel_name="France 3";}
   elsif ($service_id==0x0124) {$channel_name="France 3";}
   elsif ($service_id==0x0125) {$channel_name="Fr3 Orléans";}
   elsif ($service_id==0x0126) {$channel_name="France 3";}
   elsif ($service_id==0x0127) {$channel_name="France 3";}
   elsif ($service_id==0x0128) {$channel_name="France 3";}
   elsif ($service_id==0x0129) {$channel_name="France 3";}
   elsif ($service_id==0x012A) {$channel_name="France 3";}
   elsif ($service_id==0x012B) {$channel_name="France 3";}
   elsif ($service_id==0x012C) {$channel_name="France 3";}
   elsif ($service_id==0x012D) {$channel_name="France 3";}
   elsif ($service_id==0x012E) {$channel_name="France 3";}
   elsif ($service_id==0x012F) {$channel_name="France 3";}
   elsif ($service_id==0x0130) {$channel_name="France 3";}
   elsif ($service_id==0x0131) {$channel_name="France 3";}
   elsif ($service_id==0x0132) {$channel_name="Fr3 Dijon";}
   elsif ($service_id==0x0133) {$channel_name="Fr3 Montpellier";}
   elsif ($service_id==0x0134) {$channel_name="France 3";}
   elsif ($service_id==0x0135) {$channel_name="France 3";}
   elsif ($service_id==0x0136) {$channel_name="France 3";}
   elsif ($service_id==0x0137) {$channel_name="France 3";}
   elsif ($service_id==0x0138) {$channel_name="Fr3 Provence";}
   elsif ($service_id==0x0139) {$channel_name="France 3";}
   elsif ($service_id==0x013A) {$channel_name="Fr3 Champardenne";}
   elsif ($service_id==0x013B) {$channel_name="France 3";}
   elsif ($service_id==0x0143) {$channel_name="France 3 num 22 a";}
   elsif ($service_id==0x0144) {$channel_name="France 3 num 22 b";}
   elsif ($service_id==0x0170) {$channel_name="Locale num 20";}
   elsif ($service_id==0x0171) {$channel_name="Locale num 21";}
   elsif ($service_id==0x0172) {$channel_name="Locale num 22";}
   elsif ($service_id==0x0173) {$channel_name="Locale num 23";}
   elsif ($service_id==0x0174) {$channel_name="Locale num 24";}
   elsif ($service_id==0x0175) {$channel_name="Locale num 25";}
   elsif ($service_id==0x0176) {$channel_name="France Ô Ile de France";}
   elsif ($service_id==0x0201) {$channel_name="Direct 8";}
   elsif ($service_id==0x0203) {$channel_name="BFM TV";}
   elsif ($service_id==0x0204) {$channel_name="i > TELE";}
   elsif ($service_id==0x0205) {$channel_name="Virgin 17";}
   elsif ($service_id==0x0206) {$channel_name="GULLI";}
   elsif ($service_id==0x0207) {$channel_name="France 4";}
   elsif ($service_id==0x0301) {$channel_name="Canal+";}
   elsif ($service_id==0x0302) {$channel_name="Canal+ cinema";}
   elsif ($service_id==0x0303) {$channel_name="Canal+ sport";}
   elsif ($service_id==0x0401) {$channel_name="M6";}
   elsif ($service_id==0x0402) {$channel_name="W9";}
   elsif ($service_id==0x0403) {$channel_name="NT1";}
   elsif ($service_id==0x0404) {$channel_name="Paris Premiere";}
   elsif ($service_id==0x0407) {$channel_name="Arte HD";}
   elsif ($service_id==0x0501) {$channel_name="TF1 HD";}
   elsif ($service_id==0x0502) {$channel_name="France 2 HD";}
   elsif ($service_id==0x0503) {$channel_name="M6 HD";}
   elsif ($service_id==0x0601) {$channel_name="TF1";}
   elsif ($service_id==0x0602) {$channel_name="NRJ12";}
   elsif ($service_id==0x0603) {$channel_name="LCI";}
   elsif ($service_id==0x0604) {$channel_name="Eurosport France";}
   elsif ($service_id==0x0605) {$channel_name="TF6";}
   elsif ($service_id==0x0606) {$channel_name="TMC";}
   elsif ($service_id==0x0607) {$channel_name="Arte ";}
   elsif ($service_id==0x0801) {$channel_name="Locale num 20";}
   elsif ($service_id==0x0802) {$channel_name="Locale num 21";}
   elsif ($service_id==0x0803) {$channel_name="Locale num 22";}
   elsif ($service_id==0x0804) {$channel_name="Locale num 23";}
   elsif ($service_id==0x0805) {$channel_name="Locale num 24";}
   elsif ($service_id==0x0806) {$channel_name="Locale num 25";}
   elsif ($service_id==0x0883) {$channel_name="France 3 num 23 a";}
   elsif ($service_id==0x304) {$channel_name="Planete";}
   elsif ($service_id==0x305) {$channel_name="Canal J";}
   elsif ($service_id==0x306) {$channel_name="TPS STAR";}

   return $channel_name;
   }

sub PrintInfoEvent # ($table_id,$current_next_indicator,$running_status,$event_name)
   {
   my $table_id = $_[0];
   my $current_next_indicator = $_[1];
   my $running_status = $_[2];
   my $event_name = $_[3];
   my $result="EIT";
   
   if ($table_id == 0x4e) {$result .=" p/f actual -";}
   elsif ($table_id == 0x4f) {$result .=" p/f other -";}
   elsif (($table_id >= 0x50) and ($table_id <= 0x5f) ) {$result .=" schedule actual -";}
   elsif (($table_id >= 0x60) and ($table_id <= 0x6f) ) {$result .=" schedule other -";}
   else {$result .=" unknown -";}
   
   if ($current_next_indicator == 1) {$result .=" applic. now -";}
   elsif ($current_next_indicator == 0) {$result .=" applic. next -";}
   else {$result .=" FAULT -";}
   
   if ($running_status == 0) {$result .=" rst undefined -";}
   elsif ($running_status == 1) {$result .=" rst not running -";}
   elsif ($running_status == 2) {$result .=" rst about to start -";}
   elsif ($running_status == 3) {$result .=" rst pausing -";}
   elsif ($running_status == 4) {$result .=" rst running -";}
   elsif ($running_status == 5) {$result .=" rst off air -";}
   else {$result .=" rst rfu -";}
   
   $result .= "> ". $event_name;
   
   return $result;   
   
   }

sub PrintHexaArray
   {
   my $convhex = unpack('H*',pack('C*',@_));
   print $convhex,"\n";
   }

sub PrintHexaBuffer
   {
   my $convhex = unpack('H*',$_[0]);
   print $convhex,"\n";
   }
   
while (read (FSOURCE, $buffer, 188))
   {
   # TS Header => true for all TS packets
   @tsheader = unpack 'C4',$buffer;
   my $sync_byte = $tsheader[0];
   my $transport_error_indicator = ($tsheader[1]&0b1000_0000)/0b1000_0000;
   my $payload_unit_start_indicator = ($tsheader[1]&0b0100_0000)/0b0100_0000;
   my $transport_priority = ($tsheader[1]&0b0010_0000)/0b0010_0000;
   my $PID =  ($tsheader[1]*0x100 + $tsheader[2])&0b0001_1111_1111_1111;
   my $transport_scrambling_control = ($tsheader[3]&0b1100_0000)/0b0100_0000;
   my $adaptation_field_control = ($tsheader[3]&0b0011_0000)/0b0001_0000;
   my $continuity_counter= ($tsheader[3]&0b0000_1111);
   # print "Current packet header: 0x";
   # &PrintHexaArray(@tsheader);
   
   # check of sync_byte presence ... if not, there is an issue with the file = quit
   if (!($sync_byte==0x47)) {die("Corrupted file source or endianess issue (please then replace H by h, C by c, etc. in perl script)\n");}
   
   # we only work on EIT => PID 0x12
   if ($PID == 0x12)
      {
      if (($payload_unit_start_indicator == 0b1) and ($adaptation_field_control == 0b01)) #we don't want to dig in adaptation field yet
         {# this is a first TS packet of an EIT, no continuity_check needed
         
         # if there are any previous EIT packet, time to deal with it ...
         if ($EITPackets >0)
            {
            #############################################
            # do whatever needed with current EIT packet
            #############################################
            # print "EIT try $EITPackets gives:\n";
            # print "\$#EIT: ",$#EIT," (0x",sprintf ("%X",$#EIT),")\n";
            # print "Length from EIT[1]&[2]:",sprintf ("%u",($EIT[1]*0x100+$EIT[2])&0b0000_1111_1111_1111)," (0x",sprintf ("%04X",($EIT[1]*0x100+$EIT[2])&0b0000_1111_1111_1111),")\n";
            # &PrintHexaArray(@EIT);
            # print "Without padding 0xFF octets:\n";
            # $#EIT = (($EIT[1]*0x100+$EIT[2])&0b0000_1111_1111_1111) + 3 - 1;
            # &PrintHexaArray(@EIT);
            # print "-------------------------------------------------\n\n";
            
            #############################################
            # Let's try the parsing of array @EIT ...
            my $table_id = $EIT[0];
            my $section_syntax_indicator = ($EIT[1]&0b1000_0000)/0b1000_0000;
            my $section_length = ($EIT[1]*0x100 + $EIT[2]) & 0b0000_1111_1111_1111;
            my $service_id =  ($EIT[3]*0x100 + $EIT[4]);
            my $version_number = ($EIT[5]&0b0011_1110) / 0b0000_0010;
            my $current_next_indicator = ($EIT[5]&0b0000_0001);
            my $section_number = $EIT[6];
            my $last_section_number = $EIT[7];
            my $transport_stream_id = ($EIT[8]*0x100 + $EIT[9]);
            my $original_network_id = ($EIT[10]*0x100 + $EIT[11]);
            my $segment_last_section_number = $EIT[12];
            my $last_table_id = $EIT[13];
            my $EIT_cpt = 13;
            
            while (($EIT_cpt < ($section_length + 3 - 11)) and ($EIT_cpt<= $#EIT))
               {
               $EIT_cpt ++;
               my $event_id = $EIT[$EIT_cpt]*0x100 + $EIT[$EIT_cpt+1];
               my $start_time = ((($EIT[$EIT_cpt+2] * 0x100 + $EIT[$EIT_cpt+3])*0x100 + $EIT[$EIT_cpt+4])*0x100 + $EIT[$EIT_cpt+5])*0x100 + $EIT[$EIT_cpt+6];
               my $start_timeMJD = ($EIT[$EIT_cpt+2] * 0x100 + $EIT[$EIT_cpt+3]);
               my $start_timeBCD = ($EIT[$EIT_cpt+4]*0x100 + $EIT[$EIT_cpt+5])*0x100 + $EIT[$EIT_cpt+6];
               my $duration = ($EIT[$EIT_cpt+7] * 0x100 + $EIT[$EIT_cpt+8]) * 0x100 + $EIT[$EIT_cpt+9];
               my $running_status = (0b1110_0000 & $EIT[$EIT_cpt+10]) / 0b0010_0000;
               my $free_CA_mode = (0b0001_0000 & $EIT[$EIT_cpt+10]) / 0b0001_0000;
               my $descriptors_loop_length = (0b0000_1111 & $EIT[$EIT_cpt+10]) * 0x100 + $EIT[$EIT_cpt+11];

               # calculate the MJD time
               my $MJD_Yprime = int ( ($start_timeMJD - 15078.2) / 365.25 ); #printf "Y': $MJD_Yprime\n";
               my $MJD_Mprime = int ( ($start_timeMJD - 14956.1 - int ($MJD_Yprime * 365.25) ) / 30.6001 );  #printf "M': $MJD_Mprime\n";
               my $MJD_Day = $start_timeMJD - 14956 - int ($MJD_Yprime * 365.25) - int ($MJD_Mprime * 30.6001);  #printf "Day': $MJD_Day\n";
               my $MJD_K = 0;
               if (($MJD_Mprime == 14) or ($MJD_Mprime == 15)) {$MJD_K = 1;}
               # printf "K: $MJD_K\n";
               my $MJD_Year = $MJD_Yprime + $MJD_K + 1900;  #printf "Year: $MJD_Year\n";
               my $MJD_Month = $MJD_Mprime - 1 - $MJD_K * 12; #printf "Month: $MJD_Month\n";
               my $MJD_Date = sprintf ("%04u", $MJD_Year)."-".sprintf ("%02u", $MJD_Month)."-".sprintf ("%02u", $MJD_Day);
               my $BCD_Time = sprintf("%02u",$EIT[$EIT_cpt+4])."-".sprintf("%02u",$EIT[$EIT_cpt+5])."-".sprintf("%02u",$EIT[$EIT_cpt+6]);
               
               my $EIT_desc_cpt = $EIT_cpt+11;
               
               while ($EIT_desc_cpt < ($EIT_cpt+11+$descriptors_loop_length-1))
                  {
                  $EIT_desc_cpt ++;
                  my $descriptor_tag = $EIT[$EIT_desc_cpt];
                  my $descriptor_length = $EIT[$EIT_desc_cpt+1];
                  my @descriptor_data; #[0..($descriptor_length-1)];
                  my $desc_element; my $desc_cpt =0;
                  foreach $desc_element (@EIT[($EIT_desc_cpt+2)..($EIT_desc_cpt+1+$descriptor_length)])
                     {
                     $descriptor_data[$desc_cpt]=$desc_element;
                     $desc_cpt ++;
                     }
                  ######################################
                  # DO THINGS HERE with descriptor_data
                  #
                  if (($descriptor_tag == 0x4D) and (($EIT_desc_cpt+1+$descriptor_length)<$section_length + 3)) #we got it whole
                     {
                     #&PrintHexaArray(@descriptor_data); print pack('C*',@descriptor_data),"\n";
                     my $ISO_639_language_code = pack ('C3',shift (@descriptor_data),shift (@descriptor_data),shift (@descriptor_data));
                     my $event_name_length = shift (@descriptor_data);
                     my $event_cpt = 0;
                     my $event_name = "";
                     my $event_name_first_char = $descriptor_data [0];
                     while ($event_cpt < $event_name_length)
                        {
                        $event_cpt ++;
                        $event_name .= pack ('C*',shift (@descriptor_data));
                        }
                     
                     my $text_length = shift (@descriptor_data);
                     my $text_cpt = 0;
                     my $text = "";
                     my $text_first_char = $descriptor_data[0];
                     while ($text_cpt < $text_length)
                        {
                        $text_cpt ++;
                        $text .= pack ('C*',shift (@descriptor_data));
                        }
                     
                     # warning, following check makes the assumption that text is pre-coded correctly ... this is not always the case
                     if (!(($event_name_first_char <= 0xFF) and ($event_name_first_char >= 0x20))) #check whether we should use a local character code page
                        { #specific character code page
                        # first, we remove the first char
                        my @temp_event_str = split(//,$event_name);
                        shift (@temp_event_str);
                        #now we have to clean each char and replace it with an english one (or underscore) ... can be hard stuff
                        # my $temp_event_str_cpt = 0;
                        # while ($temp_event_str_cpt<= $#temp_event_str)
                           # {
                           # my $check_evt = unpack('C*',$temp_event_str[$temp_event_str_cpt]);
                           # $temp_event_str[$temp_event_str_cpt] = &HexToLetter($check_evt);
                           # $temp_event_str_cpt ++;
                           # }
                        #now we can recreate the target name
                        $event_name = join('',@temp_event_str);
                        }
                     
                     
                     # print results + limit them to certain type of EIT if needed (uncomment right line or none)
                     if ($table_id == 0x4e) # p/f actual
                     # if ($table_id == 0x4f) # p/f other
                     # if (($table_id >= 0x50) and ($table_id <= 0x5f)) # schedule actual
                     # if (($table_id >= 0x60) and ($table_id <= 0x6f)) # schedule other
                     # following if is out of standard ... in case
                     # if (!(($table_id == 0x4e) or ($table_id == 0x4f) or (($table_id >= 0x50) and ($table_id <= 0x5f)) or (($table_id >= 0x60) and ($table_id <= 0x6f))))
                        {
                        my $cleaned_event_name = &SimpleText ($event_name);
                        print $event_name."\/".$cleaned_event_name."\n";
                        print &PrintInfoEvent ($table_id,$current_next_indicator,$running_status,"\n$cleaned_event_name ($MJD_Date at $BCD_Time on ".&NameOfSid($service_id)." [sid:$service_id\\0x".sprintf("%02X",$service_id)."])\n");
                        print "Event_name_first_char =0x",sprintf ("%02X",$event_name_first_char)," (".pack('C*',$event_name_first_char).")\n\n";
                        }
                     
                     # print "event_name: $event_name \n"; #this is the name we want with _ instead of specific chars, if first char is correct
                     # print "proposed_name from EIT $EITPackets: $event_name-$MJD_Date-at-$BCD_Time-on-$service_id\n\n";
                        
                     # if (!(($text_first_char <= 0xFF) and ($text_first_char>=0x20))) #check whether we should use a local character code page
                        # { #specific character code page
                        # print "text: $text \n\n"; #still needs to be cleaned
                        # }
                        # else
                        # { #generic character code page
                        # print "text: $text \n\n";
                        # }
                        
                     }
                  #
                  # END OF DOING THINGS :)
                  ######################################
                  $EIT_desc_cpt += 1+$descriptor_length;
                  }
               $EIT_cpt += 11+$descriptors_loop_length;
               }
            
            #############################################
            # prepare next packet operation
            #############################################
            $#EIT = -1; # we reset the array
            }
         
         $EITPackets ++;
         $previous_continuity_counter = $continuity_counter; # for next packet checks
         $TABLErebuilding =1;
         @tspacket = unpack 'C*',$buffer;
         shift (@tspacket);
         shift (@tspacket);
         shift (@tspacket);
         shift (@tspacket);
         my $pointer_field = shift (@tspacket);
         if (!($pointer_field == 0x00))
            {
            die ();
            $TABLErebuilding =0;
            $#EIT = -1;
            }
         else
            {
            my $temp_cpt =0;
            my $element;
            foreach $element (@tspacket)
               {
               $EIT [$temp_cpt] = $element;
               $temp_cpt ++;
               }
            }
         #############################################
         #print each EIT TS packet, whether continuous, good or bad, etc.
         # my $file_offset = sprintf "%X",($packet_ref * 188);
         # print "EIT TS 1st packet ($packet_ref - offset: 0x$file_offset) from try $EITPackets - continuity: $continuity_counter\n";
         # print "TS packet is:\n";
         # &PrintHexaBuffer($buffer);
         }
      elsif (($TABLErebuilding == 1) and ($adaptation_field_control == 0b01))
         {# this is a continuing TS packet of an EIT and we have a precedent TS packet that could match + no adaptation field
         #check continuity counter
         if ((($previous_continuity_counter + 0b1)&0b0000_1111) == $continuity_counter)
            {# we can keep appending data
            $previous_continuity_counter = $continuity_counter; # for next packet checks
            @tspacket = unpack 'C*',$buffer;
            shift (@tspacket);
            shift (@tspacket);
            shift (@tspacket);
            shift (@tspacket);
            my $EIT_offset = $#EIT + 1;
            my $temp_cpt =0;
            my $element;
            foreach $element (@tspacket)
               {
               $EIT [$EIT_offset + $temp_cpt] = $element;
               $temp_cpt ++;
               }
            }
         else
            {#discontinuity
            $TABLErebuilding =0;
            $#EIT = -1;
            $EITPackets --;
            }
         #############################################
         #print each EIT TS packet, whether continuous, good or bad, etc.
         # my $file_offset = sprintf "%X",($packet_ref * 188);
         # print "EIT TS packet ($packet_ref - offset: 0x$file_offset) from try $EITPackets - continuity: $continuity_counter\n";
         # print "TS packet is:\n";
         # &PrintHexaBuffer($buffer);
         }
      }
   $#tsheader = -1;
   $#tspacket = -1;
   $packet_ref ++;
   }

close (FSOURCE);

die ("End of operation\n");

woww, very good!!
Lets see if I found some more free time, to start implementing some ideas ...

So far, the execution of your program is fine:
over my Gormitis.ts sample we get:
[victor@diablo ts]$ perl ts_2nov.pl Gormitis.ts > salida2.txt
End of operation

salida2.txt is: http://dl.dropbox.com/u/684543/varios/ms450h/Matthelas/salida2.txt

I'll have to wait to go home to access your file

The formula for the time is wrong, it should be: (replace formating: %02u is not right for BCD formatting)

Código:

               my $BCD_Time = sprintf("%02x",$EIT[$EIT_cpt+4])."-".sprintf("%02x",$EIT[$EIT_cpt+5])."-".sprintf("%02x",$EIT[$EIT_cpt+6]);


Same thing for duration if you need it.

For Spain, all the channels name will be wrong or unknown, since I hardcoded them (if you use the fonction). Sorry for that. Ideally we should get two other tables to automate this:
SDT & NIT
That + the EIT, and we should have most of the information we need ... but, this begins to look like a DTT receiver on its own :-/

I have taken again this thread.
And today I have realised a very interesting thing.
I have detected that our recorded .ts files have many empty frames, between 10% and 20%.
So, I have done a preliminary testing on a .ts file, eliminating the empty frames, and the size of the file has been reduce 15%

The resulting file, is played exactly the same as the original.

Hi!

vic1972 escribió:I have taken again this thread.
And today I have realised a very interesting thing.
I have detected that our recorded .ts files have many empty frames, between 10% and 20%.
So, I have done a preliminary testing on a .ts file, eliminating the empty frames, and the size of the file has been reduce 15%

That's unexpected. I would have thought that the LG only retained the TS actually filled with something related to the programs you are watching (i.e. linked to your PID/SID). Or I missed something in the MPEG-2 TS standard (that's possible).

The resulting file, is played exactly the same as the original.

That's a good thing. The only reason I would see to need such empty frames is to keep the "pace" for the feeding of the decoder by filling inter-burst of VBR coded streams. So, it does not need that when decoding. Perhaps it needs it when dumping/recording. Or I missed something.

BTW, how did you identify that these frames where empty? There are some other frames you could also try to drop depending on user needs after recording: subtitles, extra audio, EIT, etc.

Cheers,
Matt

These are called padding packets in the standard , to help in the syncro.

Of course we need to investigate more,
the app is here:
http://dl.dropbox.com/u/684543/varios/ms450h/Aplicaciones/ts_reductor/ts_reductor

call like:
ts_reductor origin.ts destination.ts
regards

vic1972 escribió:These are called padding packets in the standard , to help in the syncro.

Ouf, I heard about these

So we are lucky that these are not needed by DvdPlayer when playing the file

Thanks for the code!

Cheers,
Matt

Finally we do not have such great number of empty packets,
it seems that the first sample I took, was in the advertisement moment, probalby that was the cause.
but anyway, i have just taken a movie.ts, and the results are clear:
File parsed succesfully!
* Info
- Total number of packets: 3750913
- Total number of packets with information: 3748868
- Total number of empty packets: 2045
- Percentage of used packets: 99%
- Percentage of empty packets: 0%
+ No Adaptation Fields, all is payload: 3697316
+ Adaptation Fields are present, no payload: 37905
+ Adaptation Fields are present, and also payload: 13646

So, I think we better forget about saving space