Reboot-Loop Hunting Season...

Hola all there and welcome to the "Reboot-Loop Hunting Season".

Any of you hear or see on own eyes the "Reboot-Loop" issue. In short - sometime you turn the PVR off and after you turn it on again, it will not normally boot but endless reboots with messages on display "HELLO" and "WAIT".

So please, if anyone have this issue, please write here as many information as you know. Your configuration, time from you flashed the firmware, running services and all what came in your mind.
If someone don't want to quick repair it (by reflashing the firmware) but really hunt the wrong-doer and will not mind the PVR is useless for some time, he can try to contact me (please use private message of this forum or my ICQ) to try to recover some information from this "big-black-expensive-brick-still-rebooting-brick".

Last information:
- if you remove the HDD, the reboot-loop issue persist, so it is nothing about the started custom services (Samba, web server or any other), so it seems there is something overwritten which prevent the system start - any needed file from writable part of system (bookmarks, channel definition file, recording schedules, history schedules)

I would like to thank you for our entire team...

Keltek escribió:[...]welcome to the "Reboot-Loop Hunting Season".

Hi there! I'm glad you have taken it seriously. I am sure that yes, with the help of all, "we can"!

[...]if anyone have this issue, please write here as many information as you know.

Just as I have said here:

evr escribió:[...] I used the official firmware MS400_091209_0118.img and MS400_101027_0128.img for more than one month without any problem; then I decided to test Harmony and in less then 24 hours I got two "Hello-wait-reboot" oops. After the second reisntall, I completely stopped all the extra services of Custom-KV and this lasted for 3 days before the next "Hello-Wait-reboot" loop. Then I installed this special firmware (official firmware + PM3.5beta -- no Custom-KV) and, after 10 days, yesterday night, the "Hello-Wait-reboot" loop happened again (this morning I have just gone back to strictly official LG firmware MS400_101027_0128.img to see how long it works O.K.) [...]

On the other hand, regarding your comment:

[...] if you remove the HDD, the reboot-loop issue persist, so it is nothing about the started custom services (Samba, web server or any other), so it seems there is something overwritten which prevent the system start - any needed file from writable part of system (bookmarks, channel definition file, recording schedules, history schedules)[...]

Here is my intuition: Some of the added/newer software starts asking memory up to the point that the system starts using the swap (I have never seen a significant amount of used swap using only official firmware). The small (32KByts) swap partition in flash overflows and something is wrong with swap management that makes the overflowed swap invade other flash disk areas which are critically needed for booting (the kernel for example . Hence the impossibility to boot and the endless "Hello-Wait-reboot" loop... Just my 2 cents

Best,

evr escribió:[...]Here is my intuition: Some of the added/newer software starts asking memory up to the point that the system starts using the swap (I have never seen a significant amount of used swap using only official firmware). The small (32KByts) swap partition in flash overflows and something is wrong with swap management that makes the overflowed swap invade other flash disk areas which are critically needed for booting [...]

There is an easy way to check this intuition: lower the swappiness kernel parameter. It is set by default to 90 (%), meaning that the system will prefer to use swap than the cached memory. We can set it to a very low value, or better to 0, thereby preventing the system to use the swap alltogether:

Código:

echo 0 > /proc/sys/vm/swappiness

Maybe this causes some application to crash, but if my "swap overflow" theory is correct, the essential flash disk area corruption will no longer happen and maybe we can finally say "goof bye" to the infamous "Hello-wait-reboot" oops.... My second 2 cents

Best,

Thank you for your 4 cents
The link with /mnt/rd/swap.img occurred to me too. Very interesting is what you write:

Then I installed this special firmware (official firmware + PM3.5beta -- no Custom-KV) and, after 10 days, yesterday night, the "Hello-Wait-reboot" loop happened again (this morning I have just gone back to strictly official LG firmware MS400_101027_0128.img to see how long it works O.K.)

Does it mean it can do something with PipeManagement and not with other packages (recompilation of uClibc, busybox and all the stuff in "custom")?
So what about some tweaking of PM - I remember your message about piping the input/output of PM/DvdPlayer so we should rework this part.

BTW: I have created new tool-chain, recompile the whole stuff and now after 9 hours here is this status:

Código:

root@Venus:~# uptime
 23:47:31 up  9:27, load average: 0.06, 0.05, 0.01
root@Venus:~# free
            total        used        free      shared      buffers
Mem:        250728      185608        65120            0        53592
-/+ buffers:            132016      118712
Swap:      1209240            0      1209240
root@Venus:~#

Now I run lighttpd with php and transmission (ad additional klogd and syslogd with network logging).

Keltek escribió:[...]The link with /mnt/rd/swap.img occurred to me too. [...]

I think the swappiness idea should be tested as soon as possible. I can not do it myself right now since I have decided to run strictly official firmware for a few weeks to try to validate the hypothesis that the Hello-Wait oops don't happen with off. firmw. I have tried "echo 0 > /proc/sys/vm/swappiness" (with the off. firmw.) and it does not seem to cause any problem (but this means nothing with the off. firmw. because, as I have said in other messages, with off. firmw, I have never seen a swap usage greater than 0 and -- neither the Hello-Wait-reboot loop).

Very interesting is what you write: [...]
Does it mean it can do something with PipeManagement and not with other packages (recompilation of uClibc, busybox and all the stuff in "custom")?

I have also mentioned in other messages (my be in Spanish, sorry) that I fear the problem is not caused by a single component, but by a combination of components. That is, I think LG engineers have shrank the amount of physical memory up to the minimum that ensures a stable work of their DvdPlayer application along with the other minimal components included in their official firmware. Probably they have never tested the swap, because it seems to be not needed by their setting. So now, as we want to run more and hevier applications, swap is needed; and the more we add, the higher the probability of having to use some swap at one moment or the other...

So what about some tweaking of PM - I remember your message about piping the input/output of PM/DvdPlayer so we should rework this part.[..]

Yes, it was in a couple of messages: this and this (in Spanish, sorry again). It should be easy to make this change and I hope this will increase robustness and maybe interaction responsiveness.

But I do not think this will improve the situation with respect to the Hello_Wait oops. I am (slowly) reviewing the code of PipeManagement and so far I have not seen anything that might cause a large demand of memory which could result in the system using the swap. I am now focusing on a the (too many) system() calls used. All of them require the execution of busybox, but this should not be a problem since busybox is already in memory because of the "tail -f command_sender" (and I hope the busybox code is reentrant).

Best,

evr escribió:I think the swappiness idea should be tested as soon as possible. [...]

Please read this post.

Best,

I make some tests with swaps.
I remove the /mnt/rd/swap.img swap definition and keep the /dev/scsi/host0/bus0/target0/lun0/part2 swap. Then change the swappiness to 20

Código:

/sbin/sysctl -w vm.swappiness=20

in /usr/local/etc/rcS

After this I have about 70MB of free RAM.
Next I add 2 torrents to Transmission (11.5GB and 6.5GB).
Instantly the system was under heavy load - it is normal for torrents.
After some time, the memory was deplated (actualy to 3.5MB)

Código:

            total        used        free      shared      buffers
Mem:        250728      246908        3820            0          540
-/+ buffers:            246368        4360
Swap:      160640            0      160640

The system is running ok - DvdPlayer react to RC (slowly, but react), all processes is running - for now it is about 3 hours.

I have an idea - the reboot-loop issue is the /mnt/rd/swap.img
Coz it is a first swap (priority -1), system can use it anytime he wants. Because the vm.swappiness is by default to 90 (on RHEL this value is 60), the system can easily swap. So there can be request for 'make a swapping and use the /mnt/rd/swap.img'. Kernel do the swap (1byte is enough) to this swap destination which overwrite the data in swap. After you turns PVR off and on, the DvdPlayer found this swap.img damaged and do reboot loop.

So try this:
- in /usr/local/etc/rcS

Código:

...
/bin/mknod /dev/tty3 c 3 7
/bin/mknod /dev/tty4 c 4 7

# Set the swappiness
/sbin/sysctl -w vm.swappiness=20

# Export PATH
PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin
export PATH
...

- in /usr/local/etc/starthddservices.sh

Código:

...
        /bin/mkdir -p /tmp/run

        # Join swap
        /sbin/swapon /dev/scsi/host1/bus0/target0/lun0/part2 2>/dev/null
        /sbin/swapoff mnt/rd/swap.img 2>/dev/null

        # Make a backup directory
        mkdir -p /tmp/hdd/root/backup
...


Make a reboot and check the responsiveness of system (should be better) and the what the system will do under heavy load and memory usage.

you both are gurus
Good work in all your finding!!
I feel like we are in good hands.
Also I am reading and learning with this thread and the swappiness topic.

... After next 7 hours the PVR stops reacting (but system still works) ...
... must make some other tests ...

Keltek escribió:[...]I have an idea - the reboot-loop issue is the /mnt/rd/swap.img
Coz it is a first swap (priority -1), system can use it anytime he wants. Because the vm.swappiness is by default to 90 (on RHEL this value is 60), the system can easily swap. So there can be request for 'make a swapping and use the /mnt/rd/swap.img'. Kernel do the swap (1byte is enough) to this swap destination which overwrite the data in swap. After you turns PVR off and on, the DvdPlayer found this swap.img damaged and do reboot loop. [...]

I'm glad we agree almost completely! this is what posted in this same thread a few days ago:

evr escribió:Here is my intuition: Some of the added/newer software starts asking memory up to the point that the system starts using the swap (I have never seen a significant amount of used swap using only official firmware). The small (32KByts) swap partition in flash overflows and something is wrong with swap management that makes the overflowed swap invade other flash disk areas which are critically needed for booting (the kernel for example . Hence the impossibility to boot and the endless "Hello-Wait-reboot" loop... Just my 2 cents

The only difference is that do not think "1byte is enough", as you say. I think a flash corruption happens because of a flash swap overflow (i.e., >32Kbytes), which for some unknown reason is not correctly handled by the kernel.

Keltek escribió:... After next 7 hours the PVR stops reacting (but system still works) ...
... must make some other tests ...

Have you checked /proc/meminfo? How much used flash you had? Have you checked "ps" -- that is, whether the DedPlayer was still alive or had already disappear?

This evening, I had the LG set completely swapless, while at the same time testing hd_idle (see here):

Código:

hd-idle -a sda -i 600 -l /tmp/hdd/volumes/HDD1/tmp/hd-idle.log

Then I've tried to awake the disk by navigating with the IR remote through the REC directory. After a few seconds, I've heared the disk to actually spin up, but DvdPlayer has hang. In my telnet session everything was O.K., but "ps" revealed that the DvdPlayer has died. Of course, after reboot, everything is now back to normal.

Lesson learned: contrary to what I was thinking, probably it is not always possible to safely run DvdPlayer on a completely swapless system.

I will do some more tests, but I think we should at least keep the HD swap partition, as you propose. BTW, I commented in another message (I don't remember where) that, given that DvdPlayer can in theory reach virtual memory demands up to 800Mbytes, it would be a good idea to provide that amount of swap (preferably in a single partition); not just the 160 Mbytes currently activated...

Best,

I have just got a system hung (in "OFF") while simply trying to copy (with busybox "cp") a 2GB file from REC to another HDD1 directory. This was with both swaps activated. Interestingly, this has hapened just after having reached a free memory slightly below 30MB. See details this message (in Spanish). I could recover with reset without problem (I remind that I am using strictly official firmware).

Best,

evr escribió:[...]Lesson learned: contrary to what I was thinking, probably it is not always possible to safely run DvdPlayer on a completely swapless system. I will do some more tests, but I think we should at least keep the HD swap partition, as you propose. [...]

Bad news. I have tried a couple of times to just browse through the MEDIA (HDD) menus with (only) the /mnt/rd/swap.img deactivated and it seems that the LG systematically hangs. Can you please try it an report your results?

Best,

evr escribió:Bad news. I have tried a couple of times to just browse through the MEDIA (HDD) menus with (only) the /mnt/rd/swap.img deactivated and it seems that the LG systematically hangs.

Confirmed. It hangs systematically if the /mnt/rd/swap.img swap is deactivated; either the HD swap is activated or not.

However, if swappiness is set to 0 (which should be equivalent to completely deactivating all swaps), th LG seems to work flawless.

Best,

The only difference is that do not think "1byte is enough", as you say. I think a flash corruption happens because of a flash swap overflow (i.e., >32Kbytes), which for some unknown reason is not correctly handled by the kernel.

If you take a look at it by cat /proc/swaps, you can see the usage of all swaps and you can see the "/mnt/rd/swap.img" is not used. So I think in some circumstances kernel use this swap. But DvdPlayer needs this file for something weird (I don't know for what).

I'm sorry I can't write more information about freeze after 7 hours. But now it is 15 hours running without problems. After next freeze I try to collect all what can be.

I prefer to set swappiness to 60 and deactivate the "/mnt/rd/swap.img". I have created 1GB file in /tmp/hddmedia/lg and activated is as next swap space after partition. So I'll see what happen.
We can go to remove the reboot-loop problem, but went into reboot 1 time after 24 hours
As Victor said before, I now understand why LG turns the PVR off after recording

Today morning I try to watch one movie (to check I need to remux in mkvtoolnix to disable headers of tracks compression) and the DvdPlayer was killed. Here is the dmesg I get:

Código:

[AUDIO WARNING] <4><APP> Reset cgms_change_flag when APP flush
<APP> switch focus: 0x00000001 -> 0x00000000
[AUDIO WARNING] <4><APP> Reset cgms_change_flag when APP flush
HDMI_gen_audio_infoframe
ClassifyBonding  DD  0x00008280
New Format!!  0x00000001, <4>MPEG
mpg pcm ch  0x00000002 sb  0x00000020 sr  0x0000bb80 mono  0x00000000
mpg info ns  0x00000480 br  0x000000c0 layer  0x00000002
[DAC  0x0000bb80 ->  0x0000bb80]
HDMI_gen_audio_infoframe
HDMI_gen_audio_infoframe
WARNING[4]:<4>TVE p-underflow @ line 0
WARNING[2]:<4>TVE i-underflow @ line 0
HDMI_gen_audio_infoframe
HDMI_gen_audio_infoframe
<APP> switch focus: 0x00000000 -> 0xfffffffe
VideoConfig called:
1. start remap DVR zone...
No swap area dedicated for remap...
Break instruction in kernel code in arch/mips/kernel/traps.c::do_bp, line 694[#1]:
Cpu 0
$ 0  : 00000000 10007f00 00000027 8f527edc
$ 4  : 8047a718 8047a718 00000013 804d4aec
$ 8  : 8f4ae428 a0000000 a0000000 a0000000
$12  : 80000000 00000001 ffffffff 89cc727c
$16  : 804e0000 ffffffff 80543f80 00000001
$20  : 804d3c00 00000000 8047c710 00000000
$24  : 00000000 8011c4cc                 
$28  : 8e7f4000 8e7f5d30 8e7f5d78 8019828c
Hi    : 00000000
Lo    : 0000000a
epc  : 8019828c get_swap_page+0x74/0x578    Tainted: PF   
ra    : 8019828c get_swap_page+0x74/0x578
Status: 10007f03    KERNEL EXL IE
Cause : 58808024
PrId  : 00019378
Modules linked in: udf ufsd ohci_hcd ehci_hcd sata_mars libata venus_ir_new
Process LoadMedia (pid: 353, threadinfo=8e7f4000, task=8e74a7f0)
Stack : 00000010 00000001 0000002c 8047bf34 80543f80 806e14c0 80543f80 00000001
        804d3c00 00000002 8047c710 00000000 8e7f5db8 801974f0 8047c1bc 00000002
        8047c710 00000000 8e7f5db8 80184bb8 00000064 806e14c0 80543f80 8047c2a8
        80185958 8018675c 00000000 10000000 8e74a7f0 8047c2b0 8047c1bc 804d3c00
        00000064 00000000 8e7f5db8 8e7f5db8 80543f80 00000001 80543f98 8047c2a8
        ...
Call Trace:
 [<801974f0>] add_to_swap+0x40/0x1d0
 [<80184bb8>] remap_alloc_page+0x28/0x34
 [<80185958>] remap_onepage+0x210/0x1aa0
 [<8018675c>] remap_onepage+0x1014/0x1aa0
 [<801876ec>] remap_DVR_zone+0x504/0x9f4
 [<80187d2c>] start_remap+0x150/0x178
 [<80182c44>] record_delete+0x50/0xc0
 [<8018327c>] auth_ioctl+0x398/0x13e0
 [<801c89e4>] sys_ioctl+0x494/0x4f0
 [<8014dd30>] sys_setpriority+0x338/0x36c
 [<80117b00>] stack_done+0x20/0x3c
 [<80117b00>] stack_done+0x20/0x3c
 [<801c8550>] sys_ioctl+0x0/0x4f0


Code: 3c048043  0c04e162  24846d1c <0200000d> 8e02c870  8c420000  30420003  24030003  10430004
eth0 Regs : 0xb8016000
 00 e0 91 a9 f4 29 00 00 00 00 00 80 00 00 00 00
 c3 c8 3f 16 02 86 00 00 00 00 00 00 00 00 00 00
 2b 12 14 05 00 00 00 00 02 84 00 00 00 00 00 00
 00 00 00 00 00 00 00 00 00 00 00 02 03 f5 00 01
 00 00 0c 00 00 00 00 0e 00 00 00 00 00 00 00 00
 00 00 00 00 00 00 00 00 e0 00 00 00 84 01 78 6d
 08 0a 04 00 00 00 00 00 00 00 00 00 00 00 00 00
 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
eth0 Regs : 0xb8016100
 a0 4f 34 00 00 08 00 00 00 00 00 00 00 00 00 00
 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 a0 4f 30 00 00 18 02 00 00 00 00 00 00 00 00 00
eth0 Regs : 0xb8016200
 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 12 00 10 00 00 0c 34 0c 00 00 00 00 00 00 00 00
 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
eth0 RxFDP : 0xa04f3000
 00 08 00 80 10 e0 9f 0d 00 00 00 00 ff ff 00 00
 00 08 00 80 10 20 da 0a 00 00 00 00 ff ff 00 00
 00 08 00 80 10 10 54 0e 00 00 00 00 ff ff 00 00
 00 08 00 80 10 30 15 09 00 00 00 00 ff ff 00 00
 00 08 00 80 10 b0 95 0f 00 00 00 00 ff ff 00 00
 00 08 00 80 10 20 36 08 00 00 00 00 ff ff 00 00
 00 08 00 80 10 30 58 0b 00 00 00 00 ff ff 00 00
 00 08 00 80 10 60 de 0a 00 00 00 00 ff ff 00 00
 00 08 00 80 10 c0 58 09 00 00 00 00 ff ff 00 00
 00 08 00 80 10 d0 23 09 00 00 00 00 ff ff 00 00
 00 08 00 80 10 e0 a9 0b 00 00 00 00 ff ff 00 00
 00 08 00 80 10 c0 5b 0d 00 00 00 00 ff ff 00 00
 00 08 00 80 10 e0 0b 0d 00 00 00 00 ff ff 00 00
 00 08 00 80 10 30 04 0c 00 00 00 00 ff ff 00 00
 00 08 00 80 10 50 d7 0c 00 00 00 00 ff ff 00 00
 00 08 00 80 10 d0 2b 08 00 00 00 00 ff ff 00 00
 00 08 00 80 10 d0 db 0f 00 00 00 00 ff ff 00 00
 00 08 00 80 10 e0 97 0e 00 00 00 00 ff ff 00 00
 00 08 00 80 10 f0 b6 0b 00 00 00 00 ff ff 00 00
 46 40 01 32 10 f0 48 0a 22 b4 00 00 ff ff 00 00
 40 40 01 32 10 90 02 08 22 b4 00 00 ff ff 00 00
 46 40 01 32 10 b0 70 09 22 b4 00 00 ff ff 00 00
 7e 40 01 32 10 40 ef 09 22 b4 00 00 ff ff 00 00
 57 40 01 32 10 50 13 0e 22 b4 00 00 ff ff 00 00
 7e 40 01 32 10 e0 22 0c 22 b4 00 00 ff ff 00 00
 00 08 00 80 10 c0 47 0b 00 00 00 00 ff ff 00 00
 00 08 00 80 10 40 ba 0a 00 00 00 00 ff ff 00 00
 00 08 00 80 10 d0 f5 0b 00 00 00 00 ff ff 00 00
 00 08 00 80 10 a0 74 09 00 00 00 00 ff ff 00 00
 00 08 00 80 10 80 57 0f 00 00 00 00 ff ff 00 00
 00 08 00 80 10 a0 94 08 00 00 00 00 ff ff 00 00
 00 08 00 80 10 c0 c5 0c 00 00 00 00 ff ff 00 00
 00 08 00 80 10 30 5f 0f 00 00 00 00 ff ff 00 00
 00 08 00 80 10 80 63 0b 00 00 00 00 ff ff 00 00
 00 08 00 80 10 60 2a 08 00 00 00 00 ff ff 00 00
 00 08 00 80 10 a0 96 0b 00 00 00 00 ff ff 00 00
 00 08 00 80 10 50 98 0a 00 00 00 00 ff ff 00 00
 00 08 00 80 10 60 3e 09 00 00 00 00 ff ff 00 00
 00 08 00 80 10 f0 6d 08 00 00 00 00 ff ff 00 00
 00 08 00 80 10 90 9f 0d 00 00 00 00 ff ff 00 00
 00 08 00 80 10 10 d5 0f 00 00 00 00 ff ff 00 00
 00 08 00 80 10 40 cb 0d 00 00 00 00 ff ff 00 00
 00 08 00 80 10 50 cf 09 00 00 00 00 ff ff 00 00
 00 08 00 80 10 a0 70 09 00 00 00 00 ff ff 00 00
 00 08 00 80 10 10 2d 09 00 00 00 00 ff ff 00 00
 00 08 00 80 uf  0 ba 0c 00 00 00 00 ff ff 00 00
 00 08 00 80 10 b0 b9 09 00 00 00 00 ff ff 00 00
 00 08 00 80 10 a0 4b 0d 00 00 00 00 ff ff 00 00
 00 08 00 80 10 90 67 0e 00 00 00 00 ff ff 00 00
 00 08 00 80 10 80 02 08 00 00 00 00 ff ff 00 00
 00 08 00 80 10 80 52 0a 00 00 00 00 ff ff 00 00
 00 08 00 80 10 90 b9 0c 00 00 00 00 ff ff 00 00
 00 08 00 80 10 d0 f3 0e 00 00 00 00 ff ff 00 00
 00 08 00 80 10 d0 84 0c 00 00 00 00 ff ff 00 00
 00 08 00 80 10 90 08 0c 00 00 00 00 ff ff 00 00
 00 08 00 80 10 40 4c 09 00 00 00 00 ff ff 00 00
 00 08 00 80 10 80 2c 0d 00 00 00 00 ff ff 00 00
 00 08 00 80 10 70 36 0b 00 00 00 00 ff ff 00 00
 00 08 00 80 10 80 d0 08 00 00 00 00 ff ff 00 00
 00 08 00 80 10 80 e5 09 00 00 00 00 ff ff 00 00
 00 08 00 80 10 60 0e 0c 00 00 00 00 ff ff 00 00
 00 08 00 80 10 10 5d 0b 00 00 00 00 ff ff 00 00
 00 08 00 80 10 70 8e 0f 00 00 00 00 ff ff 00 00
 00 08 00 c0 10 30 ad 0e 00 00 00 00 ff ff 00 00
note: LoadMedia[353] exited with preempt_count 1
SIGTERM received.
SIGTERM received.


I don't understand the "No swap area dedicated for remap...". Does it mean the DvdPlayer will use some dedicated swap to remap DVR zone... WTF?

In traps.c on line 694 is: (line 694 is the line with die_if_kernel())

Código:

#ifdef CONFIG_REALTEK_WATCHPOINT
   } else {
      // in kernel mode...
      pepc = (unsigned long *)regs->cp0_epc + ((regs->cp0_cause & CAUSEF_BD) != 0);
      bcode = ((*pepc >> 6) & ((1 << 20) - 1));
      if (bcode != BRK_WATCH)
         die_if_kernel("Break instruction in kernel code", regs);

      opcode = *pepc;
   }
#endif

Keltek escribió:Today morning I try to watch one movie (to check I need to remux in mkvtoolnix to disable headers of tracks compression) and the DvdPlayer was killed.

Well this is exactly what I reported in my latest two messages of this thread!.

Here is the dmesg I get: [...] I don't understand the "No swap area dedicated for remap...". Does it mean the DvdPlayer will use some dedicated swap to remap DVR zone... WTF?

It must be something like that... I think there is a system call for remapping; see man 2 mremap but I didn't know its relation with swap. Swap areas can be used for peculiar things; for instance, suspend to disk uses swap area to store a copy of the current physical memory contents before the system goes down. But I must admit I am mostly ignorant about these matters.

Best,

Just an idea ok? probably I'm talking nonsense

Looking that swap seems to be needed by dvdplayer , if not reboot.
would it be a good idea in the start up to format some way the swap space, to make sure it is ready for its usage ...
regards

vic1972 escribió:Just an idea ok? probably I'm talking nonsense ;) Looking that swap seems to be needed by dvdplayer , if not reboot. would it be a good idea in the start up to format some way the swap space, to make sure it is ready for its usage ...

May be; but my intuition is different: It is not the flash swap what gets corrupted; the problem is that in critical situations under (relatively) heavy load, it overflows and somehow the swap manager fails to continue writing on the other disk swap area. Instead, it continues writing in the non-swap flash memory physically adjacent to the 32MB chunk mounted as flash, thereby corrupting some flash-resident files which are critical for booting; for instance, the kernel.

But, of course, this is only a (very week) theory...

Best,

I make some new tests and try this:

- the swap /mnt/rd/swap.img must be activated
- I activate /dev/scsi/host1/bus0/target0/lun0/part2 swap partition manually with priority "1" by

Código:

/sbin/swapon -p 1 /dev/scsi/host1/bus0/target0/lun0/part2

in /usr/local/etc/starthddservices.sh
- I tune the swapping before any swapping partition is activated by

Código:

# Set the swappines
/sbin/sysctl -w vm.swappiness=90


in top of /usr/local/etc/rcS

Now I have started all possible services (lighttpd, transmission, samba, minidlna) with following info
free

Código:

root@Venus:~# free
            total        used        free      shared      buffers
Mem:        250728      247108        3620            0        2364
-/+ buffers:            244744        5984
Swap:      1209240        9072      1200168


meminfo

Código:

root@Venus:~# cat /proc/meminfo
MemTotal:      250728 kB
MemFree:          3612 kB
Buffers:          1652 kB
Cached:        134420 kB
SwapCached:      3448 kB
Active:          49512 kB
Inactive:        94660 kB
HighTotal:          0 kB
HighFree:            0 kB
LowTotal:      250728 kB
LowFree:          3612 kB
SwapTotal:    1209240 kB
SwapFree:      1200148 kB
Dirty:          26416 kB
Writeback:          0 kB
Mapped:          12792 kB
Slab:            11612 kB
CommitLimit:  1334604 kB
Committed_AS:    25892 kB
PageTables:        696 kB
VmallocTotal:  1048548 kB
VmallocUsed:      1944 kB
VmallocChunk:  1046380 kB

and swaps

Código:

Filename                                Type            Size    Used    Priority
/mnt/rd/swap.img                        file            32      0      -1
/dev/scsi/host1/bus0/target0/lun0/part2 partition      160640  9084    1

System is now stable (for about 1h) so I keep it running for a longer time and try to play some movies...

Hello again.
I make a changes according last post with still positive results...

Código:

root@Venus:~# uptime
 23:27:06 up 2 days,  7:35, load average: 0.14, 0.12, 0.10

Código:

root@Venus:~# free
            total        used        free      shared      buffers
Mem:        250728      247072        3656            0        4892
-/+ buffers:            242180        8548
Swap:      1209240        7368      1201872

Código:

root@Venus:~# cat /proc/meminfo
MemTotal:      250728 kB
MemFree:          3536 kB
Buffers:          4896 kB
Cached:        141872 kB
SwapCached:      3876 kB
Active:          22556 kB
Inactive:      134764 kB
HighTotal:          0 kB
HighFree:            0 kB
LowTotal:      250728 kB
LowFree:          3536 kB
SwapTotal:    1209240 kB
SwapFree:      1201872 kB
Dirty:              20 kB
Writeback:          0 kB
Mapped:          19100 kB
Slab:            9272 kB
CommitLimit:  1334604 kB
Committed_AS:    25060 kB
PageTables:        708 kB
VmallocTotal:  1048548 kB
VmallocUsed:      1944 kB
VmallocChunk:  1046380 kB

Código:

root@Venus:~# cat /proc/swaps
Filename                                Type            Size    Used    Priority
/mnt/rd/swap.img                        file            32      0      -1
/dev/scsi/host1/bus0/target0/lun0/part2 partition      160640  7368    1
/tmp/hdd/volumes/HDD1/lg/swapspace      file            1048568 0      -2

Código:

root@Venus:~# sysctl vm.swappiness
vm.swappiness = 90

If you want to try it do this:

Código:

cd /usr/local/etc
wget http://www.fozona.cz/LG/reboot-loop/rcS -O rcS
wget http://www.fozona.cz/LG/reboot-loop/starthddservices.sh -O starthddservices.sh

#AND DO NOT FORGET TO CHANGE THESE FILES AS EXECUTABLE!! or you'll be doomed ;)

chmod 755 rcS starthddservices.sh

After reboot please check of you have the partition swap file with priority 1 and swappiness ( cat /proc/sys/vm/swappiness ) to 90.

User Mario from Czech contact me with reboot-looped PVR and we start to debug it... using ICQ

So we found the cause of reboot-loop problem

In the 20-seconds of booting between reboot the Maria connect to PVR and rename the /usr/local/etc/rcS to rcS1. It will prevent to start normally and use the "emergency" script /etc/init.d/rcS_emergency.
After it, the PVR normally boots - so it means no database files are damaged... and we found corrupted PipeManagement binary.
You can get the damaged binary here - http://www.fozona.cz/LG/reboot-loop/PipeManagement.BAD - and you can see the first 196608 bytes (0x30000 in hex) is cleared to zero. So we replace it with new PipeManagement and the system is fully repaired
So...
- we can check of damaged PipeManagement and if it is damaged, boot without it
- if PM is damaged, we can try it to repair by downloading from internet

Keltek escribió:User Mario from Czech contact me with reboot-looped PVR and we start to debug it... using ICQ :) So we found the cause of reboot-loop problem :) In the 20-seconds of booting between reboot the Maria connect to PVR and rename the /usr/local/etc/rcS to rcS1. It will prevent to start normally and use the "emergency" script /etc/init.d/rcS_emergency. After it, the PVR normally boots - so it means no database files are damaged... and we found corrupted PipeManagement binary. [...] the first 196608 bytes (0x30000 in hex) is cleared to zero. [...]

This is a great finding! indead! However, we still don't know what caused the corruption in the first pace...

So...
- we can check of damaged PipeManagement and if it is damaged, boot without it
- if PM is damaged, we can try it to repair by downloading from internet

O.K. for these turnarounds. But they are based on a single observation of the problem... Without understanding the corruption origin, who knows which piece of software, may be different from PM, will be corrupted the next time?

If I understand it well, in the installed Harmony Pack, PM sits in the sda4 partition of the hard disk. This is interesting, as it contradicts my theory that the problem is caused by an overflow of the flash swap into another, non-swap flash areas... One question: did you try to e2fsck the sda4 filesystem? may be there are other corrupted files...

BTW, how are you dong with your current swap setup (with higher priority for the hard disk swap)? still working without problems? I am also running with this setup without any problem so far (but I am still with official firmware, so this just means that increasing the priority of the HD swap does not causes regression).

Best,

PM is not installed on sda4 but in /usr/local/etc which is /dev/mtdblock/2. We are unable to move it to sda4.
It is very weird because any other files are untouched. Why is damaged only the PM binary?

My setup was running about 100 hours and then because of the SCSI errors I must restart.

Keltek escribió:PM is not installed on sda4 but in /usr/local/etc which is /dev/mtdblock/2. We are unable to move it to sda4.

You are right! I thought it was the HD because it is mounted R/W... So my "flash swap overflow" theory still holds... ;)

It is very weird because any other files are untouched. Why is damaged only the PM binary?

Probably just bud luck (in this particular case). This is why I predict that next time it can happen with another soft piece...

My setup was running about 100 hours and then because of the SCSI errors I must restart.

Those scsi errors is another thing we have to look at more carefully (so far I have not observed them)...

Best,