PipeMangemente integrated in sourceforge SVN

evr escribió:

Next point, the system touch command, will this be executed the same way as the wget, suging exeOut ..?

Nope ;) It is just direct syscall; I think just open() and close(). I will look in more detail to see if there is a more direct call.

I guess you need touch just to change the last accessed time of an existing file (i.e., you don't need the full touch functionality which may create an empty file). If this is the case, touch is mplemented just using utime():

Código:

int utime(const char *filename, const struct utimbuf *times);
struct utimbuf {
              time_t actime;      /* access time */
              time_t modtime;      /* modification time */
          };


Best,

And evr, what abou make the busybox as shared?
I try to make a test compilation of shared busybox. It creates a stripped binary "busybox" of size 4412 bytes and stripped shared library "libbusybox.so.1.18.2" of size 999568 bytes. So this should prevent loading more "instances" of full busybox binary.

Keltek escribió:And evr, what abou make the busybox as shared?
I try to make a test compilation of shared busybox. It creates a stripped binary "busybox" of size 4412 bytes and stripped shared library "libbusybox.so.1.18.2" of size 999568 bytes. So this should prevent loading more "instances" of full busybox binary.

But this way you move the problem to the shared library! Remember we have just 265MB of physical memory and half of it is ususally taken by the kernel plus DvdPlayer!. This shared lib would be stick in resident memory as long as a single busybox instance is running. How much resident memory takes it? ...

256MB is very little as compared with any modern PC -- And swap is great, but **not miracle** ...

This is how I see a stable, robust and responsive customized MS450:

a) PM is *very* useful and it is small and light enough to run without problems along side with DvdPlayer - except for its dependence of busybox. This is why I'm trying to get completely rid of this dependence.

b) Busybox is too demanding to have it continuously lodged in memory; It is O.K. for telnet work, administration, etc., but *not* as a fundamental piece of an active multimedia system for regular interactive use. And (much) less so with modern busybox versions. If we hit the need of some functionality which is not available in older and lighter busybox versions, we should rather write small programs or functions specifically for these functionalities, than using newer, ever larger and heavier busybox versions. For example, it is better to use a specific, small wget program than a newer busybox that carry wget.

c) The web frontend is *extreemly* useful, but I think it it should get rid of mysql, because it takes too many resources. It could be something hybrid: the control of configuration parameters and services should (obviously) work on the LG, but all the processes involving significant batabase operations, such as channels ordering, recordings programming etc., should be moved to the PC, working with copies of the corresponding DBs (I think all this can be done as a web application and can be made transparent to the user, but I admit it is not easy and I do not really know how to do it).

d) Torrent or similar hard downloading services are nice, but very demanding. They should be run only as a background activity, when the LG is idle of playing and/or interactive (menu) manipulations. This could be worked out by monitoring the load and signaling the server process with SIGSTOP/SIGCONT.

e) The Samba server is also quite demanding and I do not see it very useful, because moving a movie with the available LAN speed is not very practical (it is much faster to use an auxiliary usb disk). Moreover you can copy with the LG Samba client, which is waaaay less demanding. In any case, the Samba server should be also run only as a background process...

f) A DLNA server is also a nice addition, but again it is demanding. As in d) and e), it could be activated as a background service, when no interactive or playing activity is going on on the LG.

g) Other smaller services should be O.K, but not too many at the same time.

O.K. I've already written to much! -- so for sure there will be many non-senses in what I've said... just brainstorming...

Best wishes,

Nice post!!!

OK, I confirm wget is now working fine, and so, we have eliminated more system calls.

more elimiantions are fine.
no more system cp, and no more chmod.

resting are , if i recall well, ln , grep and call to sqlite3, which i think it will remain as a bash script.

vic1972 escribió:more elimiantions are fine. :) no more system cp, and no more chmod.

Great!

resting are , if i recall well, ln , grep and call to sqlite3, which i think it will remain as a bash script.

I've just made a svn update of your current PM code.

Your "ln -f f1 f2" call can be trivially implemented just as unlink("f2"); link("f1", "f2"); (see man link unlink).

Since your use of grep is just string matching (no regexp), it can be easily implemented by reading the whole bookmark.db (small) file into a buffer and using simple functions from the string lib (man 3 string). I can do it if you like -- please let me know.

Finally, sqlite3 should be executed as in the case of wget, using fork and execv.

BTW, as I see you do not use the contents of the buffer that collects the chilld's stdout+stderr output, the code can be simplified. For instance to call wget:

Código:

...
/*-----------------------------------------------------------------------
  mySystem: runs a "server" in argv[0] with arguments in argv[1], ...
*/
int mySystem(char *argv[]) {
  int status; pid_t pid=-1;                              /* child's PID */
 
  if((pid = fork()) == (pid_t)0) {                /* This is the child  */
    execv(argv[0], argv);                  /* child becomes the server */
    return errno;              /* we can reach here only if execv fails */
  }
  else {                                          /* This is the father */
    if((wait(&status)<0) || status) return -1; /* waits for child's end */
    kill(pid,SIGKILL);
    return status;
  }
}
/*--------------------------------------------------------------------- */
  ...
  char *cmd[] = {WGET_LG, gral_buffer, URL, (char *)0 };
  if (mySystem(cmd)) fprintf(stderr, "** error!\n");
  ...


Best,

vic1972 escribió:[...]I will check how users can collaborate in the svn directly, I guess it is only enought to have a valid source-forge username. Please provide me with your id; or try to do a checkout, then modify some comments, and try to upload the changes with your sourceforge id.

evr escribió:[...]Gracias. Espera que intente hacer un commit de prueba (aunque creo que para eso hay que tener permisos de dev). [...]

I have tried today and it does work for me (I just added a comment to PipeManagement.c, then made a successful commit and then removed the comment and made another successful commit, leaving it as it was originally).

So, if you like, for simple changes (to eliminate system() calls, for the moment) I can just make them directly to your code, rather than posting here examples of who to do... Please let me know if this is O.K. with you, or you prefer to continue doing it yourself.

Best,

evr escribió:[...]Since your use of grep is just string matching (no regexp), it can be easily implemented by reading the whole bookmark.db (small) file into a buffer and using simple functions from the string lib (man 3 string). I can do it if you like -- please let me know.[...]

[...] if you like, for simple changes (to eliminate system() calls, for the moment) I can just make them directly to your code, rather than posting here examples of who to do... Please let me know if this is O.K. with you, or you prefer to continue doing it yourself.

O.K. While waiting for your reply, here you have searchString() to replace your system() calls needed for grep, along with a simple invocation example:

Código:

#include <stdio.h>
#include <stdlib.h>
#include <fcntl.h>
#include <string.h>

int searchStr(char *string, char *file) {
  const int N=32767; int M, NN=N, found=0; int fd; char *buf, *pbuf;

  fd = open(file, O_RDONLY);  buf = malloc(N+1);  pbuf = buf;

  while((M = read(fd, pbuf, N)) == N)
    {buf = realloc(buf, NN+=N); pbuf+=N;}  pbuf[M] = '\0';
  if(strstr(buf, string)) found = 1; free(buf);

  return found;
}

int main(int argc, char *argv[]) {
  int found; char *string=argv[1], *file=argv[2];
  if(argc<3)
    {fprintf(stderr, "\nUse: %s <string> <file>\n\n", argv[0]); exit(1);}

  found = searchStr(string, file);

  printf("%sfound `%s' in %s\n", found?"":"not ", string, file);
}


Best,

Hello
sorry for late response,
has been a very interesting weekend

yes please, go ahead, and do all changes and improvements you want.
thanks very much for you help.

vic1972 escribió:[...]yes please, go ahead, and do all changes and improvements you want. thanks very much for you help.

Just for completeness, I am posting below the functions myTouch and myLink (which, in addition to searchString -- see previous message, I plan to include in PipeManagement.c), along with the corresponding small test programs.

I note, however that myLink does not work well if used on the NTFS partition. But the busybox ln does not work well either: Some files are correctly linked, but for others the link destination file has 0 bytes !!??. The other two test programs do work correctly on the LG.

So, if I have read PipeManagement.c well, there are only two types of system() calls left: those which do need a shell (to invoke scripts -- we can talk about this later) and the one which runs sqlite3. This one could be executed with fork/execv, as for wget, but as it is now it has the complication of using stdin redirection ("<"). I guess this can be avoided, but I do not know enough of sqlite to see how to do it. Vic, can you please try to reformulate the call so that it does not use stdin redirection?

Best,

Código:

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <errno.h>
#include <utime.h>
#include <sys/types.h>

int myTouch(char *path, int seconds) {
  int res=0; struct utimbuf *times;

  times = malloc(sizeof(times));
  times->actime = (time_t)seconds; times->modtime = (time_t)seconds;
  return utime(path, times);
}

int main(int argc, char *argv[]) {
  int res, timeSeconds=atol(argv[2]); char *path=argv[1];
  if(argc!=3)
    {fprintf(stderr, "\nUse: %s <path> <timeInSeconds>\n\n", argv[0]); exit(1);}

  res = myTouch(path, timeSeconds);

  printf("Res = %d\n", res);
}


Código:

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <errno.h>

int myLink(char *oldPath, char *newPath) {
  unlink(newPath); if(link(oldPath, newPath)) return errno; return 0;
}

int main(int argc, char *argv[]) {
  int res; char *oldpath=argv[1], *newpath=argv[2];
  if(argc<3)
    {fprintf(stderr, "\nUse: %s <oldPath> <newPath>\n\n", argv[0]); exit(1);}

  res = myLink(oldpath, newpath);

  printf("Res = %d\n", res);
}

evr escribió:[...]I note, however that myLink does not work well if used on the NTFS partition. But the busybox ln does not work well either: Some files are correctly linked, but for others the link destination file has 0 bytes !!??. [...]

It is quite strange: both with busybox ln and with myLink. If you try to (hard-)link an old file (one you created yesterday, for instance) both link programs fail, creating a 0-bytes file, apparently with 2 linls according to ls -l (but the original file remains with 1 link). Now if you just touch the original file, then both link programs do work correctly, creating a "clone" file of the same size as the original one and both files with the right 2 links.

Vic, have you observed this weired behavior?

Best,

WOWW!!!!
Good finding!!

It is happening also to me:

Código:


-rwxrwxrwx    1 root    root    14952022016 Jan 21 01:49 Sof??a_20110119_2205.ts
-rwxrwxrwx    1 root    root    14855958528 Jan 28 15:20 Sof??a_20110126_2223.ts
root@Venus:/tmp/hdd/volumes/HDD1/REC# rm \ Repeat.ts
root@Venus:/tmp/hdd/volumes/HDD1/REC# ln -f Sofía_20110119_2205.ts Repeat.ts
root@Venus:/tmp/hdd/volumes/HDD1/REC# ls -l
total 45255360
-rwxrwxrwx    1 root    root    4258942976 Feb  7 23:41 Antena 3 Noticias 2_20110207_2151.ts
-rwxrwxrwx    1 root    root    789491712 Feb  8 08:02 BEN 10_20110208_0732.ts
-rwxrwxrwx    1 root    root    603525120 Feb  7 15:17 Bob Esponja_20110207_1455.ts
-rwxrwxrwx    1 root    root    5337202688 Jan 30 22:59 Los Protegidos_20110130_2158.ts
-rwxrwxrwx    1 root    root    5544345600 Feb  7 00:20 Los Protegidos_20110206_2158.ts
-r-xr-xr-x    2 root    root            0 Feb  8 18:07 Repeat.ts
-rwxrwxrwx    1 root    root    14952022016 Jan 21 01:49 Sof??a_20110119_2205.ts
-rwxrwxrwx    1 root    root    14855958528 Jan 28 15:20 Sof??a_20110126_2223.ts
root@Venus:/tmp/hdd/volumes/HDD1/REC# cat Repeat.ts
root@Venus:/tmp/hdd/volumes/HDD1/REC# rm Repeat.ts
root@Venus:/tmp/hdd/volumes/HDD1/REC# touch Sofía_20110119_2205.ts
root@Venus:/tmp/hdd/volumes/HDD1/REC# ln -f Sofía_20110119_2205.ts Repeat.ts
root@Venus:/tmp/hdd/volumes/HDD1/REC# ls -l
total 59856944
-rwxrwxrwx    1 root    root    4258942976 Feb  7 23:41 Antena 3 Noticias 2_20110207_2151.ts
-rwxrwxrwx    1 root    root    789491712 Feb  8 08:02 BEN 10_20110208_0732.ts
-rwxrwxrwx    1 root    root    603525120 Feb  7 15:17 Bob Esponja_20110207_1455.ts
-rwxrwxrwx    1 root    root    5337202688 Jan 30 22:59 Los Protegidos_20110130_2158.ts
-rwxrwxrwx    1 root    root    5544345600 Feb  7 00:20 Los Protegidos_20110206_2158.ts
-rwxrwxrwx    2 root    root    14952022016 Feb  8 18:08 Repeat.ts
-rwxrwxrwx    2 root    root    14952022016 Feb  8 18:08 Sof??a_20110119_2205.ts
-rwxrwxrwx    1 root    root    14855958528 Jan 28 15:20 Sof??a_20110126_2223.ts


I have to touch first, to make link work.
It is strange, because when working in the bookmark.db thread, I noticed this 0 file before, but I though it was due to
deleting original file...

But strange enough also, we have been using (me and George) this link with the bookmark.db thread for Repeat, and all testing
was fine so far ...

I will take a look at these.
Thanks very much for your discovering.

More things,
I have seen the little program for touch,
it have parameters of time in seconds, this is seconds from 1-1-1970 ?

if i want to issue this command: touch -d '2001-01-01 00:00:01'
how can i do this? thanks

vic1972 escribió:More things, I have seen the little program for touch, it have parameters of time in seconds, this is seconds from 1-1-1970 ? if i want to issue this command: touch -d '2001-01-01 00:00:01' how can i do this? thanks

Just:

myTouch(file, 978303601)

You can find this (or similarly any other) number of seconds using:

date -d '2001-01-01 00:00:01' +%s.

Of course, there is a syscal to convert time expressed in terms of year, month, day, hh, mm, ss into seconds since the epoch. But I think you do not need this in PM.

Best,

evr escribió:[...] if you like, for simple changes (to eliminate system() calls, for the moment) I can just make them directly to your code, rather than posting here examples of how to do... Please let me know if this is O.K. with you, or you prefer to continue doing it yourself.

vic1972 escribió:[...] yes please, go ahead, and do all changes and improvements you want. [...]

I have not touched anything since the two pending changes are both in the bookmark_db_process(), which you have been working with in the last few days (and I did not wont to add my possible bugs to your ongoing issues...). But maybe now it is already time to remove the two remaining system() calls, used for grep and sqlite (apart from those which are used to execute scripts, which need more thinking). Please let me know if you prefer me to wait a bit more before doing the changes.

For the sqlite3 call, I asked:

[...]This one could be executed with fork/execv, as for wget, but as it is now it has the complication of using stdin redirection ("<"). I guess this can be avoided, but I do not know enough of sqlite to see how to do it. Vic, can you please try to reformulate the call so that it does not use stdin redirection?

Is this possible?

Best,

yes, thanks. very busy these days

I have tested, and this is fine:

Código:


root@Venus:/usr/local/etc/dvdplayer# sqlite3 -init ./hola.sql bookmark.db
-- Loading resources from ./hola.sql
Error: near line 1: near "use": syntax error
1|DB_VERSION|1
2|DB_DEVELOP_VERSION|6
3|SETUP_INIT|1
4|RESTORE_DEFAULT_FROM_DB|0
5|0m|��_M


basycally the sql statements are put in a file: hola.sql
and then we call with -init parameters

Would this help?

vic1972 escribió:yes, thanks. very busy these days :) I have tested, and this is fine:
[...]
basycally the sql statements are put in a file: hola.sql and then we call with -init parameters
Would this help?

Yes. Thanks. I will post here a proposal before actually touching the PipeManagement code.

Here it is:

Código:

if ((result1!=0) && (result2==0)) {
  char *cmd[]={SQLITE3_CLIENT_FILE_PATH,
              "-init", PATH_FOR_TRIGGERS,
              BOOKMARK_DB_FILE_PATH, (char *)0 };
  sprintf(gral_buffer,"%s %s %s %s", cmd[1], cmd[2], cmd[3], cmd[4]);
  logger(gral_buffer,-1);
  result1=mySystem(cmd);
}

I am assuming the TRIGGERS file is the -init (SQL) command file needed. Is this O.K.?

Best,

Hola a todos, ya que los tres sabeis español podias continuar en este idioma.

Gracias.

Manolo escribió:Hola a todos, ya que los tres sabeis español podias continuar en este idioma.

No, Keltek no entiende (mucho el castellano; ver [Tienes que estar registrado y conectado para ver este vínculo]:

Keltek escribió:Hello boys why you not write in english? [...]

Pero si no te es fácil (o no te apetece) expresarte en inglés, ¡no hay problema: usa la lengua que te sea más cómoda! -- si hay algo de importancia que le interese a Keltek ya lo traduciremos alguno de nosotros... (espero que no sea en Ruso ni en Euskera )

Saludos,

evr escribió:
I am assuming the TRIGGERS file is the -init (SQL) command file needed. Is this O.K.?

yes, in a preliminary test i did, this hola.sql was executed, i suppose the installations of triggers will work the same way.
thanks

vic1972 escribió:

evr escribió:I am assuming the TRIGGERS file is the -init (SQL) command file needed. Is this O.K.?

yes, in a preliminary test i did, this hola.sql was executed, i suppose the installations of triggers will work the same way.

Thanks. I am waiting you finish with the new release before making my modifications to PipeManagement. Hope I can start soon...

BTW, I think invoking the sqlite3 binary to do what is needed in PM is abusive! We should rather implement it using direct calls to the sqlite3 library, "libsqlite3". I am totally ignorant of SQL and databeses in general, but the API for libsqlite3 is well documented, for instance at: [Tienes que estar registrado y conectado para ver este vínculo]

What do you think?

Best,

hello
yes, willing also to close version, i think we are closing in this week.

regarding the access to db with native cliente, of course, it is the best to use the library ...
please, modify the source of PM, when we close version, to use the client of library.

in the other hand, i am thinking that in reality pm does not really query a normal sql query,
it is just an installation of triggers ....
probably there is another posible solution ...
like if detecting that there is no GUID, then replace the .db file with a .db file with the triggers ....

more possible workaorund?
I think it is very good we discuss the different approaches
bye

vic1972 escribió:regarding the access to db with native cliente, of course, it is the best to use the library ... please, modify the source of PM, when we close version, to use the client of library.

No; as I said, I can not do DB programming at all. Then, for the moment, I will just modify PM only to execute sqlite3 without first invoking busybox. If you or George know how to do it using the libsqlite3 library, this can be changed later...

in the other hand, i am thinking that in reality pm does not really query a normal sql query,
it is just an installation of triggers .... probably there is another posible solution ... like if detecting that there is no GUID, then replace the .db file with a .db file with the triggers .... more possible workaorund? I think it is very good we discuss the different approaches :)

Sorry, I do not really understand what you are talking about. Hope I'll find time to learn something about SQL one of these days.

Best,

evr escribió:

vic1972 escribió:regarding the access to db with native cliente, of course, it is the best to use the library ... please, modify the source of PM, when we close version, to use the client of library.

No; as I said, I can not do DB programming at all. Then, for the moment, I will just modify PM only to execute sqlite3 without first invoking busybox. If you or George know how to do it using the libsqlite3 library, this can be changed later...

in the other hand, i am thinking that in reality pm does not really query a normal sql query,
it is just an installation of triggers .... probably there is another posible solution ... like if detecting that there is no GUID, then replace the .db file with a .db file with the triggers .... more possible workaorund? I think it is very good we discuss the different approaches

Sorry, I do not really understand what you are talking about. Hope I'll find time to learn something about SQL one of these days.

Best,

Hello evr,

let me explain the situation - LG machine itself works with db files (eg bookmark.db) by libsqlite library which is included in "monolithic" DvdPlayer, ie using its "native client". The only thing we need to do with sqlite is to add triggers into bookmark.db - but of course we cannot use native client in DvdPlayer so either we can insert triggers into existing bookmark.db (by executing of sqlite3 or direct call using libsqlite3 library) or simply replace existing (original) bookmark.db by our modified version (which already includes the triggers), so in this case it is not needed to use sqlite at all. As we need to decide whether the bookmark.db is original or modified one I added the unique ID (GUID) into db table so it is easy to check the bookmark.db version. So the question is which method of bookmark.db modification (ie trigger inserting by sqlite client/library code or whole file replacing) would be more suitable ...

George2005 escribió:let me explain the situation - LG machine itself works with db files (eg bookmark.db) by libsqlite library which is included in "monolithic" DvdPlayer, ie using its "native client". The only thing we need to do with sqlite is to add triggers into bookmark.db - but of course we cannot use native client in DvdPlayer so either we can insert triggers into existing bookmark.db (by executing of sqlite3 or direct call using libsqlite3 library) or simply replace existing (original) bookmark.db by our modified version (which already includes the triggers), so in this case it is not needed to use sqlite at all. As we need to decide whether the bookmark.db is original or modified one I added the unique ID (GUID) into db table so it is easy to check the bookmark.db version. So the question is which method of bookmark.db modification (ie trigger inserting by sqlite client/library code or whole file replacing) would be more suitable ...

Thanks for the detailed explanation! However I do not still fully understand it. As far as I can see, bookmark.db contains timestamps indicating the instant each movie has been stopped by the user. So it should regularly be "modified" because users do use the player regularly ¿am I right? On the other hand, to add the unique ID (GUID) and to check if it exists you need to do some sqlite operations, which would anyway need to use sqlite3 or libsqlite3...

So it looks like I am still confused...

Best,