It is currently Mon Sep 21, 2020 2:54 pm

View unanswered posts | View active topics


Board index » GNS3 » Development » Bug reports

All times are UTC




Post new topic Reply to topic  Page 1 of 2
  [ 19 posts ]  Go to page 1, 2  Next
  Print view Previous topic | Next topic 
Author Message
flaviojs
 Post subject: [Fixed] dynamips crash, compiled in cygwin with x86 arch
PostPosted: Sun Jun 02, 2013 7:06 pm 
Offline

Joined: Wed May 22, 2013 7:48 am
Posts: 93
Location: Portugal
I'm running GNS3 0.8.3.1 in Windows 7.

I started investigating the dynamips crashes but dynamips.exe.stackdump usually contains an empty stack.
I did get an actual stack trace once but the official exe had no debug symbols so I stopped there.

Later I started trying my own cygwin compiles (latest, so RC6) with debug symbols to try and get a stack trace again.
Here are my runs, no linux_eth, no gen_eth:
  • arch x86, CFLAGS -Wall -O3 -fomit-frame-pointer -static -static-libgcc -m32 ==> crashed (didn't measure running time), empty stack trace
  • arch x86, CFLAGS -Wall -O3 -fomit-frame-pointer -ggdb -static -static-libgcc -m32 ==> crashed (didn't measure running time), empty stack trace
  • arch x86, CFLAGS -Wall -O2 -fomit-frame-pointer -ggdb -static -static-libgcc -m32 ==> crashed after 5mins, empty stack trace
  • arch x86, CFLAGS -Wall -O1 -fomit-frame-pointer -ggdb -static -static-libgcc -m32 ==> crashed after 1h11, empty stack trace
  • arch x86, CFLAGS -Wall -O3 -fomit-frame-pointer -gdwarf-2 -fasynchronous-unwind-tables -fvar-tracking -static -static-libgcc -m32 ==> crashed after 4h07, empty stack trace
  • arch nojit, CFLAGS -Wall -O3 -fomit-frame-pointer -static -static-libgcc -m32 ==> I stopped it after about 24h
Ok, so the problem seems to be related to the arch x86 code. I can't remove -fomit-frame-pointer, which should be the cause of the empty stack traces, because of the asm code.

This time I tried with an external dynamips instance in gdb, no linux_eth, no gen_eth, arch x86, CFLAGS -Wall -O3 -fomit-frame-pointer -ggdb -fasynchronous-unwind-tables -fvar-tracking -static -static-libgcc -m32:
Code:
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 12716.0x3c4c]
ppc32_jit_tcb_apply_patches (iop=0x825d8d70, cpu=<optimized out>, block=<optimized out>)
    at /cygdrive/d/dev/hg/dynamips-community/stable/ppc32_jit.c:529
529              ppc32_jit_tcb_set_patch(jit_ptr,jit_dst);
(gdb) bt full
#0  ppc32_jit_tcb_apply_patches (iop=0x825d8d70, cpu=<optimized out>, block=<optimized out>)
    at /cygdrive/d/dev/hg/dynamips-community/stable/ppc32_jit.c:529
        pos = <optimized out>
        disp = <optimized out>
        size = <optimized out>
        jit_dst = 0x8b3f4ad8 ""
        patch = 0x833e24fc
        pos = 0
#1  ppc32_op_gen_page (cpu=0x8253ed30, b=0x82736cd0)
    at /cygdrive/d/dev/hg/dynamips-community/stable/ppc32_jit.c:1096
        tag = <optimized out>
        gcpu = 0x8253eb78
        iop = 0x825d8d70
        cur_ia = <optimized out>
        jit_ptr = 0x4 <Address 0x4 out of bounds>
        i = <optimized out>
#2  0x004303b6 in ppc32_jit_tcb_recompile (cpu=0x8253ed30, block=0x82736cd0)
    at /cygdrive/d/dev/hg/dynamips-community/stable/ppc32_jit.c:1184
No locals.
#3  0x00430a2f in ppc32_jit_tcb_exec (block=0x82736cd0, cpu=0x8253ed30)
    at /cygdrive/d/dev/hg/dynamips-community/common/ppc32_x86_trans.h:55
        jit_code = 0x0
        offset = 985
#4  ppc32_jit_tcb_run (block=0x82736cd0, cpu=0x8253ed30)
    at /cygdrive/d/dev/hg/dynamips-community/stable/ppc32_jit.c:1206
No locals.
#5  ppc32_jit_run_cpu (gen=0x8253eb78) at /cygdrive/d/dev/hg/dynamips-community/stable/ppc32_jit.c:1297
        cpu = 0x8253ed30
        timer_irq_thread = 0x824eb1f8
        block = 0x82736cd0
        timer_irq_check = 500
#6  0x610fe08a in pthread::thread_init_wrapper(void*) () from /usr/bin/cygwin1.dll
No symbol table info available.
#7  0x610874d2 in thread_wrapper(void*) () from /usr/bin/cygwin1.dll
No symbol table info available.

Then with -O0:
Code:
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 7224.0x2d38]
0x004413f6 in ppc32_jit_tcb_apply_patches (cpu=0x801d2f18, block=0x802e4830, iop=0x8024aba0)
    at /cygdrive/d/dev/hg/dynamips-community/stable/ppc32_jit.c:529
529              ppc32_jit_tcb_set_patch(jit_ptr,jit_dst);
(gdb) bt full
#0  0x004413f6 in ppc32_jit_tcb_apply_patches (cpu=0x801d2f18, block=0x802e4830, iop=0x8024aba0)
    at /cygdrive/d/dev/hg/dynamips-community/stable/ppc32_jit.c:529
        pos = 0x7e286ef4 <Address 0x7e286ef4 out of bounds>
        disp = <optimized out>
        size = 0
        patch = 0x9e770884
        jit_ptr = 0x7e286ef3 <Address 0x7e286ef3 out of bounds>
        jit_dst = 0x9e1f8bf0 ""
        pos = 0
        __FUNCTION__ = "ppc32_jit_tcb_apply_patches"
#1  0x0044294e in ppc32_op_gen_page (cpu=0x801d2f18, b=0x802e4830)
    at /cygdrive/d/dev/hg/dynamips-community/stable/ppc32_jit.c:1096
        tag = <optimized out>
        gcpu = 0x801d2d60
        iop = 0x8024aba0
        cur_ia = <optimized out>
        jit_ptr = <optimized out>
        i = 559
#2  0x00442ca6 in ppc32_jit_tcb_recompile (cpu=0x801d2f18, block=0x802e4830)
    at /cygdrive/d/dev/hg/dynamips-community/stable/ppc32_jit.c:1184
No locals.
#3  0x0044321c in ppc32_jit_tcb_exec (block=0x802e4830, cpu=0x801d2f18)
    at /cygdrive/d/dev/hg/dynamips-community/common/ppc32_x86_trans.h:55
        jit_code = 0x0
        offset = 985
#4  ppc32_jit_tcb_run (block=0x802e4830, cpu=0x801d2f18)
    at /cygdrive/d/dev/hg/dynamips-community/stable/ppc32_jit.c:1206
No locals.
#5  ppc32_jit_run_cpu (gen=0x801d2d60) at /cygdrive/d/dev/hg/dynamips-community/stable/ppc32_jit.c:1297
        cpu = 0x801d2f18
        timer_irq_thread = 0x801a1548
        block = 0x802e4830
        ia_hash = 5216
        timer_irq_check = 347
#6  0x610fe08a in pthread::thread_init_wrapper(void*) () from /usr/bin/cygwin1.dll
No symbol table info available.
#7  0x610874d2 in thread_wrapper(void*) () from /usr/bin/cygwin1.dll
No symbol table info available.

Ok, crashed in the same place. Finally, something I can report... that pos/jit_ptr having an address out of bounds is probably why it got a SIGSEGV.
I'm not sure how to proceed from here until I get a better grasp of the code, so feel free to ask for stuff to try.

PS - I didn't include the topology I used for this because dynamips crashes in pretty much any scenario with random crash times. In this scenario I was using 5 routers, 2 switches and working with OSPF+iBGP+eBGP. (tends to crash earlier that my previous scenarios, probably because it uses more routers)




Last edited by flaviojs on Sun Jun 16, 2013 12:17 pm, edited 1 time in total.

Top
 Profile  
 
flaviojs
 Post subject: Re: dynamips crash, compiled in cygwin with x86 arch
PostPosted: Mon Jun 03, 2013 9:20 am 
Offline

Joined: Wed May 22, 2013 7:48 am
Posts: 93
Location: Portugal
I looked around the code and values of the functions in the stack trace and have a generic idea of what was going on:
A particular page/block of code has been executed enough times to trigger a JIT recompilation, it produced opcodes, generated JIT code and was applying the first patch which has suspicious values (jit_insn=NULL ppc_ia=0).
Code:
(gdb) frame 1
#1  0x0044294e in ppc32_op_gen_page (cpu=0x801d2f18, b=0x802e4830)
    at /cygdrive/d/dev/hg/dynamips-community/stable/ppc32_jit.c:1096
1096                ppc32_jit_tcb_apply_patches(cpu,b,iop);
(gdb) print *gcpu->[email protected]
$70 = {0x0 <repeats 559 times>, 0x8024aba0, 0x9e783458, 0x9e781d68, 0x9e78e6d8, 0x9e77ce38, 0x9e769470,
  0x9e78ef48, 0x9e78f008, 0x9e78f218, 0x9e769860, 0x9e7689c0, 0x9e78ec78, 0x9e79cb38, 0x9e78f678,
  0x9e769050, 0x9e782668, 0x9e7961d8, 0x9e77bc08, 0x9e78f4f8, 0x9e7782f8, 0x9e75cda0, 0x9e773680,
  0x9e782bb8, 0x9e75c5c0, 0x9e78ed08, 0x9e769290, 0x9e766028, 0x9e777608, 0x9e773500, 0x9e75cdd0,
  0x9e77e198, 0x9e774670, 0x9e774700, 0x80245fb0, 0x9e77bba8, 0x9e77b998, 0x80282c38, 0x9e77caa8,
  0x9e769890, 0x802806f8, 0x9e796cb8, 0x9e7ad040, 0x9e78ebe8, 0x9e796e68, 0x8032cc28, 0x9e796388,
  0x9e797b88, 0x9e79e7c8, 0x9e796a18, 0x9e796c58, 0x9e79df58, 0x9e78f2d8, 0x9e769260, 0x9e797408,
  0x8029fe50, 0x9e527070, 0x9e796ce8, 0x9e796748, 0x9e78e888, 0x81174590, 0x9e781f78, 0x9e78efd8,
  0x9e7be548, 0x9e796a78, 0x9e79ce98, 0x9e797318, 0x9e77ba28, 0x9e796208, 0x9e797528, 0x9e796f58,
  0x9e797948, 0x9e77eca8, 0x9e797978, 0x9e796c88, 0x9e79c7a8, 0x9e79c718, 0x9e79cb08, 0x9e7939b0,
  0x9e796358, 0x9e79c778, 0x9e7697a0, 0x9e79cbc8, 0x9e7933e0, 0x9e7afa08, 0x9e794200, 0x9e797618,
  0x802a7558, 0x9e79e618, 0x805fb9b8, 0x9e79da48, 0x9e796658, 0x9e77b188, 0x9e794e00, 0x9e7d2e30,
  0x9e720f68, 0x9e79ce68, 0x9e79dd18, 0x9e79d1f8, 0x9e77e408, 0x9e7af048, 0x9e79d678, 0x9e792930,
  0x9e79de08, 0x9e79e7f8, 0x9e79ce38, 0x9e79e8b8, 0x9e79e558, 0x9e798308, 0x9e7960a8, 0x9e7ad280,
  0x9e7a91b8, 0x9e7942f0, 0x9e79d168, 0x9e781558, 0x9e79eac8, 0x9e7aeef8, 0x9e79e6d8, 0x9e73ef80,
  0x9e78e948, 0x9e794650, 0x9e7750c0, 0x802a6b48, 0x9e7a43d8, 0x9e7982a8, 0x81161798, 0x9e7980f8,
  0x9e79c658, 0x8032cf98, 0x9e732268, 0x9e794710, 0x9e793f60, 0x9e7a86d8, 0x805fc578, 0x9e768ae0,
  0x9e79d928, 0x9e7a0f28, 0x9e78e738, 0x9e7943e0, 0x9e794590, 0x9e75ca10, 0x9e794680, 0x9e79cb98,
  0x8024da50, 0x9e79e9a8, 0x9e796568, 0x9e75c770, 0x9e760718, 0x9e78f4b8, 0x8026d0e8, 0x9e7a7f88,
  0x8027b638, 0x9e792b70, 0x9e79d498, 0x80254130, 0x9e7a0e98, 0x805ff270, 0x9e7932f0, 0x9e7968f8,
  0x9e767b20, 0x9e7a4b58, 0x9e769c80, 0x9e768c00, 0x9e72aad0, 0x9e796e98, 0x9e794770, 0x9e793650,
  0x9e769fb0, 0x9e793da0, 0x9e7a4858, 0x9e79e6a8, 0x9e793110, 0x8021cb68, 0x9e7a3448, 0x9e78f368,
  0x9e797af8, 0x9e798578, 0x9e7a32f8, 0x80222208, 0x9e794b30, 0x9e7984b8, 0x9e7573c0, 0x9e7a3c58,
  0x9e7a0e38, 0x9e7a4078, 0x9e7685a0, 0x9e7a4678, 0x9e7a3f88, 0x9e7a3358, 0x9e7a38c8, 0x9e7a4348...}
(gdb) print i
$71 = 559
(gdb) frame 0
#0  0x004413f6 in ppc32_jit_tcb_apply_patches (cpu=0x801d2f18, block=0x802e4830, iop=0x8024aba0)
    at /cygdrive/d/dev/hg/dynamips-community/stable/ppc32_jit.c:529
529              ppc32_jit_tcb_set_patch(jit_ptr,jit_dst);
(gdb) print *cpu
$72 = {ia = 2174807908, gpr = {255, 2197473336, 2191261696, 2197473360, 255, 255, 0, 2200563944, 31,
    2197473484, 4294967295, 2197473488, 2147483650, 0, 2200563944, 2200630420, 16777219, 2197474816,
    2200563944, 2199717832, 0, 0, 0, 2199717832, 2197473360, 0, 2197473616, 2200695868, 2200564212, 0, 1,
    2197473912}, vtlb = {{vaddr = 4294967295, haddr = 4294967295} <repeats 32 times>}, irq_pending = 0,
  irq_check = 0, xer = 0, lr = 2155976608, ctr = 255, reserve = 0, xer_ca = 1, cr_fields = {4, 0, 0, 0, 0,
    0, 0, 2}, mts_cache = {0xfec30008, 0xfebe0008}, exec_blk_map = 0xfeb51000, exec_phys_map = 0xfeb01000,
  translate = 0x43ead1 <ppc32_translate>, mem_op_fn = {0x0, 0x0, 0x43f411 <ppc32_lbz>, 0x43f4a1 <ppc32_lhz>,
    0x43f53e <ppc32_lwz>, 0x43f6a8 <ppc32_lha>, 0x43f777 <ppc32_stb>, 0x43f80c <ppc32_sth>,
    0x43f8ad <ppc32_stw>, 0x43f5d1 <ppc32_lwbr>, 0x43f940 <ppc32_stwbr>, 0x43fa17 <ppc32_lsw>,
    0x43faeb <ppc32_stsw>, 0x43fb9d <ppc32_lfd>, 0x43fcfc <ppc32_stfd>, 0x43fe5a <ppc32_icbi>},
  mem_op_lookup = 0x43ebb1 <ppc32_mem_lookup>, mts_slow_lookup = 0x43debc <ppc32_slow_lookup>,
  irq_count = 2033816, timer_irq_count = 0, irq_fp_count = 0, irq_lock = 0x0, tcb_list = 0x0,
  tcb_last = 0x0, tcb_free_list = 0x8038d228, exec_page_area = 0xfdb00000, exec_page_area_size = 16777216,
  exec_page_count = 512, exec_page_alloc = 1, exec_page_free_list = 0x801d57c8,
  exec_page_array = 0x801d4f00, idle_pc = 2152319232, timer_irq_pending = 9, timer_irq_armed = 0,
  timer_irq_freq = 250, timer_irq_check_itv = 1000, timer_drift = 0, irq_disable = 0, bat = {{{reg = {
          4293918750, 4293918721}}, {reg = {8190, 1}}, {reg = {0, 3997040754}}, {reg = {2147491838, 1}}, {
        reg = {33554432, 0}}, {reg = {0, 0}}, {reg = {0, 0}}, {reg = {0, 0}}}, {{reg = {2147491838, 66}}, {
        reg = {8190, 42}}, {reg = {1073774590, 1073741866}}, {reg = {4026540030, 4026531882}}, {reg = {
          33554432, 0}}, {reg = {0, 0}}, {reg = {0, 0}}, {reg = {0, 0}}}}, sr = {0 <repeats 16 times>},
  sdr1 = 0, sdr1_hptr = 0x0, msr = 36914, srr0 = 2152228504, srr1 = 36914, dsisr = 0, dar = 0, sprg = {0,
    4294967295, 0, 0}, pvr = 5243394, tb = 130945050, dec = 2147483647, hid0 = 0, hid1 = 0, sw_pos = 0,
  ppc405_tlb = {{tlb_hi = 0, tlb_lo = 0, tid = 0} <repeats 64 times>}, ppc405_pid = 0,
  mpc860_immr = 1744896000, fpu = {reg = {0 <repeats 32 times>}}, gen = 0x801d2d60, vm = 0x800f9660,
  mts_misses = 34347025, mts_lookups = 8906739945, jit_flush_method = 0, compiled_pages = 12,
  fast_memop = 0, exec_blk_direct_jump = 0, njm_exec_page = 2174803968, njm_exec_ptr = 0xfb2fe000,
  perf_counter = 0, insn_exec_count = 489098030, breakpoints = {0, 0, 0, 0, 0, 0, 0, 0},
  breakpoints_enabled = 0, jit_hreg_seq_name = 0x519314 <x86_cc_signed_map+319> "addi", ppc_reg_map = {6,
    -1, -1, -1, -1, -1, -1, -1, 2, 0, -1, 1, -1 <repeats 20 times>}, hreg_map_list = 0x801d37ac,
  hreg_lru = 0x801d3810, hreg_map = {{hreg = 0, vreg = 9, flags = 0, prev = 0x801d37ac, next = 0x801d37c0}, {
      hreg = 1, vreg = 11, flags = 1, prev = 0x0, next = 0x801d3798}, {hreg = 2, vreg = 8, flags = 0,
      prev = 0x801d3798, next = 0x801d3810}, {hreg = 0, vreg = 0, flags = 0, prev = 0x0, next = 0x0}, {
      hreg = 0, vreg = 0, flags = 0, prev = 0x0, next = 0x0}, {hreg = 0, vreg = 0, flags = 0, prev = 0x0,
      next = 0x0}, {hreg = 6, vreg = 0, flags = 0, prev = 0x801d37c0, next = 0x0}, {hreg = 0, vreg = 0,
      flags = 0, prev = 0x0, next = 0x0}}}
(gdb) print *block
$73 = {start_ia = 2174803968, jit_insn_ptr = 0x9e1a2078, acc_count = 911, ppc_code = 0xfb2fe000,
  ppc_trans_pos = 1024, jit_chunk_pos = 1, jit_ptr = 0xfe4d4a3c "\304\f\273\034", jit_buffer = 0x801d58d0,
  jit_chunks = {0x0 <repeats 64 times>}, patch_table = 0x0, prev = 0x0, next = 0x803b68d8, phys_page = 6670,
  phys_hash = 35337, phys_pprev = 0x0, phys_next = 0x0, target_bitmap = {0 <repeats 30 times>, 33554432, 0},
  target_undef_cnt = 16}
(gdb) print *iop
$74 = {opcode = 1, param = {-1, -1, -1}, arg_ptr = 0x9e770884,
  insn_name = 0x519346 <x86_cc_signed_map+369> "b", next = 0x9e78c548, ob_size_index = 4,
  ob_final = 0xfe4d1abb <incomplete sequence \351>, ob_ptr = 0x8024abcd "",
  ob_data = 0x8024abc8 <incomplete sequence \351>}
(gdb) print patch
$75 = (struct ppc32_insn_patch *) 0x9e770884
(gdb) print *patch
$76 = {next = 0x9e148b40, jit_insn = 0x0, ppc_ia = 0}
(gdb) print *patch->next
$77 = {next = 0x0, jit_insn = 0x191 <Address 0x191 out of bounds>, ppc_ia = 2658601080}

My current suspicion is that an invalid patch was produced.
For now I'm gonna enable DEBUG_BLOCK_PATCH and see what is produced during a normal run, then i'll try to understand how they are generated (or supposed to be generated).
Unfortunately gdb doesn't have generate-core-file implemented in cygwin so that's all the info i'll get from this run.
Top
 Profile  
 
flaviojs
 Post subject: Re: dynamips crash, compiled in cygwin with x86 arch
PostPosted: Mon Jun 03, 2013 10:15 pm 
Offline

Joined: Wed May 22, 2013 7:48 am
Posts: 93
Location: Portugal
I found out that in a 'healthy' ppc32_op_gen_page the variable b->patch_table is not NULL in the "Apply patches and free opcodes" section, where it crashes.
In the stack traces we have a NULL b->patch_table so something clearly went wrong with it.

Inspecting further, it's NULL before the "Generate JIT opcodes" section, and not NULL after leaving that section.
The only place that changes the value of b->patch_table in that section is ppc32_jit_tcb_record_patch.
A few runs and changes later I got this nice crash: (doesn't crash applying the first patch, which is rare)
Code:
before malloc: block=0x80435c90, block->patch_table=0x0
after malloc: block->patch_table=0x82971c90
Block 0x80630000: recording patch [JIT:0x80241ea2->ppc:0x8063026c], MTP=9
Block 0x80630000: recording patch [JIT:0x8021860a->ppc:0x80630280], MTP=12
Block 0x80630000: recording patch [JIT:0x8023ce82->ppc:0x80630424], MTP=19
Block 0x80630000: recording patch [JIT:0x80245cb2->ppc:0x80630424], MTP=21
Block 0x80630000: recording patch [JIT:0x8020eb5a->ppc:0x806300e0], MTP=28
Block 0x80630000: recording patch [JIT:0x8022127a->ppc:0x80630088], MTP=30
Block 0x80630000: recording patch [JIT:0x80230f3a->ppc:0x806301c8], MTP=33
Block 0x80630000: recording patch [JIT:0x80442fda->ppc:0x806301c8], MTP=42
Block 0x80630000: recording patch [JIT:0x802473f2->ppc:0x806301c8], MTP=45
Block 0x80630000: recording patch [JIT:0x802632aa->ppc:0x8063016c], MTP=48
Block 0x80630000: recording patch [JIT:0x8026d14a->ppc:0x806301c8], MTP=51
Block 0x80630000: recording patch [JIT:0x802114da->ppc:0x806300b8], MTP=54
Block 0x80630000: recording patch [JIT:0x80218c70->ppc:0x806301c8], MTP=55
Block 0x80630000: recording patch [JIT:0x8026b49a->ppc:0x80630174], MTP=57
Block 0x80630000: recording patch [JIT:0x8026345a->ppc:0x806301c8], MTP=61
Block 0x80630000: recording patch [JIT:0x8023a1e8->ppc:0x80630130], MTP=71
Block 0x80630000: recording patch [JIT:0x8026f66a->ppc:0x8063016c], MTP=74
Block 0x80630000: recording patch [JIT:0x8026ec4a->ppc:0x80630158], MTP=77
Block 0x80630000: recording patch [JIT:0x8021feda->ppc:0x80630158], MTP=80
Block 0x80630000: recording patch [JIT:0x8021aa6a->ppc:0x8063012c], MTP=82
Block 0x80630000: recording patch [JIT:0x804440ba->ppc:0x80630120], MTP=85
Block 0x80630000: recording patch [JIT:0x8026e07a->ppc:0x806300fc], MTP=89
Block 0x80630000: recording patch [JIT:0x80231f80->ppc:0x806301c8], MTP=90
Block 0x80630000: recording patch [JIT:0x8023a0b8->ppc:0x806301cc], MTP=92
Block 0x80630000: recording patch [JIT:0x80216f70->ppc:0x806301a0], MTP=99
Block 0x80630000: recording patch [JIT:0x8021226a->ppc:0x8063016c], MTP=102
Block 0x80630000: recording patch [JIT:0x8023ccd2->ppc:0x806301c8], MTP=105
Block 0x80630000: recording patch [JIT:0x8025b882->ppc:0x806301c8], MTP=108
Block 0x80630000: recording patch [JIT:0x8026c3ca->ppc:0x8063019c], MTP=110
Block 0x80630000: recording patch [JIT:0x8021b10a->ppc:0x80630190], MTP=113
Block 0x80630000: recording patch [JIT:0x8022292a->ppc:0x80630288], MTP=116
Block 0x80630000: recording patch [JIT:0x8021bdba->ppc:0x806301f8], MTP=122
before malloc: block=0x80435c90, block->patch_table=0x82971c90
after malloc: block->patch_table=0x82975d68
Block 0x80630000: recording patch [JIT:0x8023dcb2->ppc:0x80630200], MTP=125
Block 0x80630000: recording patch [JIT:0x8023aef8->ppc:0x80630204], MTP=127
Block 0x80630000: recording patch [JIT:0x80227b5a->ppc:0x80630430], MTP=136
Block 0x80630000: recording patch [JIT:0x80265e8a->ppc:0x80630274], MTP=147
Block 0x80630000: recording patch [JIT:0x802507b2->ppc:0x8063026c], MTP=149
Block 0x80630000: recording patch [JIT:0x802587a2->ppc:0x80630430], MTP=154
Block 0x80630000: recording patch [JIT:0x805d1648->ppc:0x80630430], MTP=156
Block 0x80630000: recording patch [JIT:0x802538a2->ppc:0x80630288], MTP=159
Block 0x80630000: recording patch [JIT:0x80240988->ppc:0x80630430], MTP=161
Block 0x80630000: recording patch [JIT:0x80220f1a->ppc:0x80630424], MTP=163
Block 0x80630000: recording patch [JIT:0x8022bcba->ppc:0x80630360], MTP=175
Block 0x80630000: recording patch [JIT:0x8021f61a->ppc:0x806303cc], MTP=181
Block 0x80630000: recording patch [JIT:0x8021f7ca->ppc:0x806303b0], MTP=184
Block 0x80630000: recording patch [JIT:0x8026b2ea->ppc:0x80630350], MTP=189
Block 0x80630000: recording patch [JIT:0x80251482->ppc:0x80630350], MTP=191
Block 0x80630000: recording patch [JIT:0x80304418->ppc:0x80630328], MTP=197
Block 0x80630000: recording patch [JIT:0x801fedf2->ppc:0x806303c8], MTP=200
Block 0x80630000: recording patch [JIT:0x8026360a->ppc:0x80630350], MTP=203
Block 0x80630000: recording patch [JIT:0x8020f0da->ppc:0x80630350], MTP=206
Block 0x80630000: recording patch [JIT:0x8023222a->ppc:0x80630324], MTP=208
Block 0x80630000: recording patch [JIT:0x80242d62->ppc:0x80630318], MTP=211
Block 0x80630000: recording patch [JIT:0x8022a82a->ppc:0x806302e4], MTP=214
Block 0x80630000: recording patch [JIT:0x80220770->ppc:0x806303b0], MTP=215
Block 0x80630000: recording patch [JIT:0x80219670->ppc:0x80630388], MTP=221
Block 0x80630000: recording patch [JIT:0x8021117a->ppc:0x806303c8], MTP=224
Block 0x80630000: recording patch [JIT:0x80262d9a->ppc:0x806303b0], MTP=227
Block 0x80630000: recording patch [JIT:0x802119ea->ppc:0x806303b0], MTP=230
Block 0x80630000: recording patch [JIT:0x80ae8502->ppc:0x80630384], MTP=232
Block 0x80630000: recording patch [JIT:0x8022cd5a->ppc:0x80630378], MTP=235
Block 0x80630000: recording patch [JIT:0x802646ea->ppc:0x806303c4], MTP=237
Block 0x80630000: recording patch [JIT:0x8021de30->ppc:0x806303c8], MTP=240
Block 0x80630000: recording patch [JIT:0x80214c2a->ppc:0x80630424], MTP=244
before malloc: block=0x80435c90, block->patch_table=0x82975d68
after malloc: block->patch_table=0x8297f9e8
Block 0x80630000: recording patch [JIT:0x8026db6a->ppc:0x80630424], MTP=252
Block 0x80630000: recording patch [JIT:0x8026f15a->ppc:0x80630488], MTP=286
Block 0x80630000: recording patch [JIT:0x8023a448->ppc:0x806304c4], MTP=287
Block 0x80630000: recording patch [JIT:0x8023a7d8->ppc:0x806304c8], MTP=289
Block 0x80630000: recording patch [JIT:0x8023a6a8->ppc:0x806304b0], MTP=295
Block 0x80630000: recording patch [JIT:0x8022c85a->ppc:0x80630480], MTP=298
Block 0x80630000: recording patch [JIT:0x80211d4a->ppc:0x806304c4], MTP=301
Block 0x80630000: recording patch [JIT:0x8022d35a->ppc:0x806304a0], MTP=304
Block 0x80630000: recording patch [JIT:0x80215e1a->ppc:0x806305fc], MTP=307
Block 0x80630000: recording patch [JIT:0x80246a42->ppc:0x80630588], MTP=314
Block 0x80630000: recording patch [JIT:0x8026396a->ppc:0x806305fc], MTP=319
Block 0x80630000: recording patch [JIT:0x80ae350a->ppc:0x806305d8], MTP=322
Block 0x80630000: recording patch [JIT:0x802427b2->ppc:0x80630578], MTP=327
Block 0x80630000: recording patch [JIT:0x8022562a->ppc:0x80630578], MTP=329
Block 0x80630000: recording patch [JIT:0x80239d28->ppc:0x80630550], MTP=335
Block 0x80630000: recording patch [JIT:0x8026e3da->ppc:0x806305ec], MTP=338
Block 0x80630000: recording patch [JIT:0x802220ea->ppc:0x80630578], MTP=341
Block 0x80630000: recording patch [JIT:0x8022d76a->ppc:0x80630578], MTP=344
Block 0x80630000: recording patch [JIT:0x8098d022->ppc:0x8063054c], MTP=346
Block 0x80630000: recording patch [JIT:0x802281da->ppc:0x80630540], MTP=349
Block 0x80630000: recording patch [JIT:0x8023243a->ppc:0x8063050c], MTP=352
Block 0x80630000: recording patch [JIT:0x8028a718->ppc:0x806305d8], MTP=353
Block 0x80630000: recording patch [JIT:0x80227020->ppc:0x806305b0], MTP=359
Block 0x80630000: recording patch [JIT:0x8026c21a->ppc:0x806305ec], MTP=362
Block 0x80630000: recording patch [JIT:0x80ae7ff2->ppc:0x806305d8], MTP=365
Block 0x80630000: recording patch [JIT:0x801ff662->ppc:0x806305d8], MTP=368
Block 0x80630000: recording patch [JIT:0x8022feca->ppc:0x806305ac], MTP=370
Block 0x80630000: recording patch [JIT:0x80257392->ppc:0x806305a0], MTP=373
Block 0x80630000: recording patch [JIT:0x8098d2c2->ppc:0x806305f4], MTP=375
Block 0x80630000: recording patch [JIT:0x80241368->ppc:0x806305f8], MTP=378
Block 0x80630000: recording patch [JIT:0x80226de0->ppc:0x806305fc], MTP=380
Block 0x80630000: recording patch [JIT:0x80442aca->ppc:0x80630804], MTP=384
before malloc: block=0x80435c90, block->patch_table=0x8297f9e8
after malloc: block->patch_table=0x82987d68
Block 0x80630000: recording patch [JIT:0x8098d4d2->ppc:0x80630634], MTP=387
Block 0x80630000: recording patch [JIT:0x80214e4a->ppc:0x806307d4], MTP=395
Block 0x80630000: recording patch [JIT:0x805c7900->ppc:0x80630808], MTP=396
Block 0x80630000: recording patch [JIT:0x80263e7a->ppc:0x8063069c], MTP=400
Block 0x80630000: recording patch [JIT:0x8023d4c2->ppc:0x80630784], MTP=409
Block 0x80630000: recording patch [JIT:0x8022255a->ppc:0x80630784], MTP=412
Block 0x80630000: recording patch [JIT:0x802467d2->ppc:0x80630728], MTP=415
Block 0x80630000: recording patch [JIT:0x8026d4aa->ppc:0x80630784], MTP=418
Block 0x80630000: recording patch [JIT:0x8024c1a2->ppc:0x80630674], MTP=421
Block 0x80630000: recording patch [JIT:0x80227450->ppc:0x80630784], MTP=422
Block 0x80630000: recording patch [JIT:0x8022189a->ppc:0x80630730], MTP=424
Block 0x80630000: recording patch [JIT:0x8026beba->ppc:0x80630784], MTP=428
Block 0x80630000: recording patch [JIT:0x80229220->ppc:0x806306ec], MTP=438
Block 0x80630000: recording patch [JIT:0x8024e4c2->ppc:0x80630728], MTP=441
Block 0x80630000: recording patch [JIT:0x8023edc2->ppc:0x80630714], MTP=444
Block 0x80630000: recording patch [JIT:0x8021d6da->ppc:0x80630714], MTP=447
Block 0x80630000: recording patch [JIT:0x80246082->ppc:0x806306e8], MTP=449
Block 0x80630000: recording patch [JIT:0x80252442->ppc:0x806306dc], MTP=452
Block 0x80630000: recording patch [JIT:0x80442e2a->ppc:0x806306b8], MTP=456
Block 0x80630000: recording patch [JIT:0x80289a08->ppc:0x80630784], MTP=457
Block 0x80630000: recording patch [JIT:0x8022e6f0->ppc:0x80630788], MTP=459
Block 0x80630000: recording patch [JIT:0x805d3058->ppc:0x8063075c], MTP=466
Block 0x80630000: recording patch [JIT:0x80252652->ppc:0x80630728], MTP=469
Block 0x80630000: recording patch [JIT:0x8021b32a->ppc:0x80630784], MTP=472
Block 0x80630000: recording patch [JIT:0x802666fa->ppc:0x80630784], MTP=475
Block 0x80630000: recording patch [JIT:0x80267b3a->ppc:0x80630758], MTP=477
Block 0x80630000: recording patch [JIT:0x80242602->ppc:0x8063074c], MTP=480
Block 0x80630000: recording patch [JIT:0x802657ca->ppc:0x806307d0], MTP=483
Block 0x80630000: recording patch [JIT:0x8026603a->ppc:0x80630804], MTP=488
Block 0x80630000: recording patch [JIT:0x8021246a->ppc:0x80630804], MTP=491
Block 0x80630000: recording patch [JIT:0x80243c12->ppc:0x80630808], MTP=499
Block 0x80630000: recording patch [JIT:0x8026aa7a->ppc:0x80630804], MTP=504
before malloc: block=0x80435c90, block->patch_table=0x82987d68
after malloc: block->patch_table=0x82990e50
Block 0x80630000: recording patch [JIT:0x8021528a->ppc:0x80630804], MTP=506
Block 0x80630000: recording patch [JIT:0x80240d78->ppc:0x80630808], MTP=512
Block 0x80630000: recording patch [JIT:0x801d8442->ppc:0x80630908], MTP=527
Block 0x80630000: recording patch [JIT:0x8020f4aa->ppc:0x80630858], MTP=529
Block 0x80630000: recording patch [JIT:0x802197aa->ppc:0x806308f0], MTP=533
Block 0x80630000: recording patch [JIT:0x80249122->ppc:0x806308f0], MTP=536
Block 0x80630000: recording patch [JIT:0x8023b8e2->ppc:0x806308a4], MTP=540
Block 0x80630000: recording patch [JIT:0x80444a4a->ppc:0x806308f0], MTP=550
Block 0x80630000: recording patch [JIT:0x8023d668->ppc:0x806308f4], MTP=552
Block 0x80630000: recording patch [JIT:0x80252972->ppc:0x806308f0], MTP=555
Block 0x80630000: recording patch [JIT:0x8023f122->ppc:0x806308cc], MTP=560
Block 0x80630000: recording patch [JIT:0x8023b7a8->ppc:0x806308dc], MTP=562
Block 0x80630000: recording patch [JIT:0x802248ba->ppc:0x806308f0], MTP=568
Block 0x80630000: recording patch [JIT:0x8022c48a->ppc:0x80630834], MTP=571
Block 0x80630000: recording patch [JIT:0x802404b2->ppc:0x80630ab0], MTP=585
Block 0x80630000: recording patch [JIT:0x8023e652->ppc:0x80630940], MTP=589
Block 0x80630000: recording patch [JIT:0x802697ea->ppc:0x80630a60], MTP=591
Block 0x80630000: recording patch [JIT:0x802234fa->ppc:0x80630a0c], MTP=602
Block 0x80630000: recording patch [JIT:0x8024ea22->ppc:0x80630c38], MTP=608
Block 0x80630000: recording patch [JIT:0x802105ba->ppc:0x80630c34], MTP=611
Block 0x80630000: recording patch [JIT:0x8021e32a->ppc:0x806309fc], MTP=616
Block 0x80630000: recording patch [JIT:0x80264daa->ppc:0x806309fc], MTP=618
Block 0x80630000: recording patch [JIT:0x803e4668->ppc:0x806309d4], MTP=624
Block 0x80630000: recording patch [JIT:0x8023bf22->ppc:0x80630bdc], MTP=627
Block 0x80630000: recording patch [JIT:0x802681fa->ppc:0x806309fc], MTP=630
Block 0x80630000: recording patch [JIT:0x80243a12->ppc:0x806309fc], MTP=633
Block 0x80630000: recording patch [JIT:0x8022334a->ppc:0x806309d0], MTP=635
Block 0x80630000: recording patch [JIT:0x80217baa->ppc:0x806309c4], MTP=638
Block 0x80630000: recording patch [JIT:0x8026711a->ppc:0x80630990], MTP=641
Block 0x80630000: recording patch [JIT:0x80226a10->ppc:0x80630c34], MTP=642
Block 0x80630000: recording patch [JIT:0x8021dcb0->ppc:0x80630a34], MTP=648
Block 0x80630000: recording patch [JIT:0x8021039a->ppc:0x80630bdc], MTP=651
before malloc: block=0x80435c90, block->patch_table=0x82990e50
after malloc: block->patch_table=0x82990fe0
Block 0x80630000: recording patch [JIT:0x8022e00a->ppc:0x80630c34], MTP=654
Block 0x80630000: recording patch [JIT:0x8023c3b2->ppc:0x80630c34], MTP=657
Block 0x80630000: recording patch [JIT:0x80230b2a->ppc:0x80630a30], MTP=659
Block 0x80630000: recording patch [JIT:0x80262a3a->ppc:0x80630a24], MTP=662
Block 0x80630000: recording patch [JIT:0x80256718->ppc:0x80630c34], MTP=663
Block 0x80630000: recording patch [JIT:0x8021630a->ppc:0x80630c34], MTP=667
Block 0x80630000: recording patch [JIT:0x80224be0->ppc:0x80630a98], MTP=673
Block 0x80630000: recording patch [JIT:0x80245902->ppc:0x80630bdc], MTP=676
Block 0x80630000: recording patch [JIT:0x802637ba->ppc:0x80630c34], MTP=679
Block 0x80630000: recording patch [JIT:0x80ae31aa->ppc:0x80630a88], MTP=682
Block 0x80630000: recording patch [JIT:0x802160a0->ppc:0x80630c34], MTP=683
Block 0x80630000: recording patch [JIT:0x802131ea->ppc:0x80630acc], MTP=687
Block 0x80630000: recording patch [JIT:0x805beee8->ppc:0x80630b08], MTP=688
Block 0x80630000: recording patch [JIT:0x802315f0->ppc:0x80630b0c], MTP=690
Block 0x80630000: recording patch [JIT:0x80229940->ppc:0x80630af4], MTP=696
Block 0x80630000: recording patch [JIT:0x80252be2->ppc:0x80630ac4], MTP=699
Block 0x80630000: recording patch [JIT:0x8021d4ba->ppc:0x80630b08], MTP=702
Block 0x80630000: recording patch [JIT:0x8026762a->ppc:0x80630ae4], MTP=705
Block 0x80630000: recording patch [JIT:0x8026855a->ppc:0x80630c38], MTP=708
Block 0x80630000: recording patch [JIT:0x8023d7a2->ppc:0x80630be4], MTP=719
Block 0x80630000: recording patch [JIT:0x8021c94a->ppc:0x80630c38], MTP=724
Block 0x80630000: recording patch [JIT:0x802589b2->ppc:0x80630c34], MTP=727
Block 0x80630000: recording patch [JIT:0x8021c72a->ppc:0x80630bcc], MTP=732
Block 0x80630000: recording patch [JIT:0x8024c9b2->ppc:0x80630bcc], MTP=734
Block 0x80630000: recording patch [JIT:0x8023a908->ppc:0x80630ba4], MTP=740
Block 0x80630000: recording patch [JIT:0x8026b9aa->ppc:0x80630bdc], MTP=743
Block 0x80630000: recording patch [JIT:0x80216cea->ppc:0x80630bcc], MTP=746
Block 0x80630000: recording patch [JIT:0x8026deca->ppc:0x80630bcc], MTP=749
Block 0x80630000: recording patch [JIT:0x8026546a->ppc:0x80630ba0], MTP=751
Block 0x80630000: recording patch [JIT:0x8022b4ea->ppc:0x80630b94], MTP=754
Block 0x80630000: recording patch [JIT:0x80244012->ppc:0x80630b60], MTP=757
Block 0x80630000: recording patch [JIT:0x80240858->ppc:0x80630c34], MTP=758
before malloc: block=0x80435c90, block->patch_table=0x82990fe0
after malloc: block->patch_table=0x82991170
Block 0x80630000: recording patch [JIT:0x805d19d8->ppc:0x80630c38], MTP=760
Block 0x80630000: recording patch [JIT:0x8023b1b8->ppc:0x80630c0c], MTP=766
Block 0x80630000: recording patch [JIT:0x8021506a->ppc:0x80630bdc], MTP=769
Block 0x80630000: recording patch [JIT:0x8024a372->ppc:0x80630c34], MTP=772
Block 0x80630000: recording patch [JIT:0x80240182->ppc:0x80630c34], MTP=775
Block 0x80630000: recording patch [JIT:0x8026cc3a->ppc:0x80630c08], MTP=777
Block 0x80630000: recording patch [JIT:0x8021168a->ppc:0x80630bfc], MTP=780
Block 0x80630000: recording patch [JIT:0x8024d302->ppc:0x80630c54], MTP=783
Block 0x80630000: recording patch [JIT:0x8023f2d2->ppc:0x80630f14], MTP=785
Block 0x80630000: recording patch [JIT:0x805d4b70->ppc:0x80630f48], MTP=788
Block 0x80630000: recording patch [JIT:0x80245ab2->ppc:0x80630f1c], MTP=791
Block 0x80630000: recording patch [JIT:0x80259922->ppc:0x80630cdc], MTP=795
Block 0x80630000: recording patch [JIT:0x8021a2ea->ppc:0x80630c84], MTP=797
Block 0x80630000: recording patch [JIT:0x8022e48a->ppc:0x80630dc4], MTP=800
Block 0x80630000: recording patch [JIT:0x80212fea->ppc:0x80630dc4], MTP=809
Block 0x80630000: recording patch [JIT:0x8021c3ca->ppc:0x80630dc4], MTP=812
Block 0x80630000: recording patch [JIT:0x80219b7a->ppc:0x80630d68], MTP=815
Block 0x80630000: recording patch [JIT:0x80242252->ppc:0x80630dc4], MTP=818
Block 0x80630000: recording patch [JIT:0x80226c3a->ppc:0x80630cb4], MTP=821
Block 0x80630000: recording patch [JIT:0x8023bc38->ppc:0x80630dc4], MTP=822
Block 0x80630000: recording patch [JIT:0x80257602->ppc:0x80630d70], MTP=824
Block 0x80630000: recording patch [JIT:0x802683aa->ppc:0x80630dc4], MTP=828
Block 0x80630000: recording patch [JIT:0x80219170->ppc:0x80630d2c], MTP=838
Block 0x80630000: recording patch [JIT:0x8024e252->ppc:0x80630d68], MTP=841
Block 0x80630000: recording patch [JIT:0x8026963a->ppc:0x80630d54], MTP=844
Block 0x80630000: recording patch [JIT:0x8022f98a->ppc:0x80630d54], MTP=847
Block 0x80630000: recording patch [JIT:0x8024f8d2->ppc:0x80630d28], MTP=849
Block 0x80630000: recording patch [JIT:0x8021fd2a->ppc:0x80630d1c], MTP=852
Block 0x80630000: recording patch [JIT:0x80ae36ba->ppc:0x80630cf8], MTP=856
Block 0x80630000: recording patch [JIT:0x80240ea8->ppc:0x80630dc4], MTP=857
Block 0x80630000: recording patch [JIT:0x80219f90->ppc:0x80630dc8], MTP=859
Block 0x80630000: recording patch [JIT:0x80238ee8->ppc:0x80630d9c], MTP=866
before malloc: block=0x80435c90, block->patch_table=0x82991170
after malloc: block->patch_table=0x82991bd0
Block 0x80630000: recording patch [JIT:0x8024b8f2->ppc:0x80630d68], MTP=869
Block 0x80630000: recording patch [JIT:0x8024cf72->ppc:0x80630dc4], MTP=872
Block 0x80630000: recording patch [JIT:0x8026c8da->ppc:0x80630dc4], MTP=875
Block 0x80630000: recording patch [JIT:0x801fff72->ppc:0x80630d98], MTP=877
Block 0x80630000: recording patch [JIT:0x80a00dda->ppc:0x80630d8c], MTP=880
Block 0x80630000: recording patch [JIT:0x80250c32->ppc:0x80630f1c], MTP=883
Block 0x80630000: recording patch [JIT:0x80212c3a->ppc:0x80630f3c], MTP=887
Block 0x80630000: recording patch [JIT:0x80443f0a->ppc:0x80630eb0], MTP=898
Block 0x80630000: recording patch [JIT:0x8020f28a->ppc:0x80630f04], MTP=903
Block 0x80630000: recording patch [JIT:0x8022dc4a->ppc:0x80630f00], MTP=906
Block 0x80630000: recording patch [JIT:0x802194ca->ppc:0x80630e98], MTP=911
Block 0x80630000: recording patch [JIT:0x8023e1c2->ppc:0x80630e98], MTP=913
Block 0x80630000: recording patch [JIT:0x80224ec0->ppc:0x80630e70], MTP=919
Block 0x80630000: recording patch [JIT:0x8021b80a->ppc:0x80630ea8], MTP=922
Block 0x80630000: recording patch [JIT:0x80259612->ppc:0x80630e98], MTP=925
Block 0x80630000: recording patch [JIT:0x80255cf2->ppc:0x80630e98], MTP=928
Block 0x80630000: recording patch [JIT:0x80250a22->ppc:0x80630e6c], MTP=930
Block 0x80630000: recording patch [JIT:0x80269b4a->ppc:0x80630e60], MTP=933
Block 0x80630000: recording patch [JIT:0x8026d9ba->ppc:0x80630e2c], MTP=936
Block 0x80630000: recording patch [JIT:0x805d4580->ppc:0x80630f00], MTP=937
Block 0x80630000: recording patch [JIT:0x80227690->ppc:0x80630f04], MTP=939
Block 0x80630000: recording patch [JIT:0x805d4f00->ppc:0x80630ed8], MTP=945
Block 0x80630000: recording patch [JIT:0x8026438a->ppc:0x80630ea8], MTP=948
Block 0x80630000: recording patch [JIT:0x802272aa->ppc:0x80630f00], MTP=951
Block 0x80630000: recording patch [JIT:0x80259d42->ppc:0x80630f00], MTP=954
Block 0x80630000: recording patch [JIT:0x80267e9a->ppc:0x80630ed4], MTP=956
Block 0x80630000: recording patch [JIT:0x8026a8ca->ppc:0x80630ec8], MTP=959
Block 0x80630000: recording patch [JIT:0x80263cca->ppc:0x80630f1c], MTP=962
Block 0x80630000: recording patch [JIT:0x8021091a->ppc:0x80630c48], MTP=964
Block 0x80630000: recording patch [JIT:0x80240fd8->ppc:0x80630f48], MTP=966
Block 0x80630000: recording patch [JIT:0x80ae81a2->ppc:0x80630f44], MTP=968
Block 0x80630000: recording patch [JIT:0x80224ffa->ppc:0x80630f3c], MTP=971
before malloc: block=0x80435c90, block->patch_table=0x82991bd0
after malloc: block->patch_table=0x82995990
Block 0x80630000: recording patch [JIT:0x8023181a->ppc:0x80630f44], MTP=974
Block 0x80630000: recording patch [JIT:0x80239bf8->ppc:0x80630f48], MTP=976
Block 0x80630000: recording patch [JIT:0x8021e6fa->ppc:0x80630f94], MTP=994
Block 0x80630000: applying patch [JIT:0x80241ea2->ppc:0x8063026c=JIT:0xfe1401c7, ]
Block 0x80630000: applying patch [JIT:0x8021860a->ppc:0x80630280=JIT:0xfe140259, ]
Block 0x80630000: applying patch [JIT:0x8023ce82->ppc:0x80630424=JIT:0xfe140d86, ]
Block 0x80630000: applying patch [JIT:0x80245cb2->ppc:0x80630424=JIT:0xfe140d86, ]
Block 0x80630000: applying patch [JIT:0x8020eb5a->ppc:0x806300e0=JIT:0xfe5a7557, ]
Block 0x80630000: applying patch [JIT:0x8022127a->ppc:0x80630088=JIT:0xfe5a725c, ]
Block 0x80630000: applying patch [JIT:0x80230f3a->ppc:0x806301c8=JIT:0xfe5a7c8f, ]
Block 0x80630000: applying patch [JIT:0x80442fda->ppc:0x806301c8=JIT:0xfe5a7c8f, ]
Block 0x80630000: applying patch [JIT:0x802473f2->ppc:0x806301c8=JIT:0xfe5a7c8f, ]
Block 0x80630000: applying patch [JIT:0x802632aa->ppc:0x8063016c=JIT:0xfe5a79d9, ]
Block 0x80630000: applying patch [JIT:0x8026d14a->ppc:0x806301c8=JIT:0xfe5a7c8f, ]
Block 0x80630000: applying patch [JIT:0x802114da->ppc:0x806300b8=JIT:0xfe5a73c4, ]
Block 0x80630000: applying patch [JIT:0x80218c70->ppc:0x806301c8=JIT:0xfe5a7c8f, ]
Block 0x80630000: applying patch [JIT:0x8026b49a->ppc:0x80630174=JIT:0xfe5a79e6, ]
Block 0x80630000: applying patch [JIT:0x8026345a->ppc:0x806301c8=JIT:0xfe5a7c8f, ]
Block 0x80630000: applying patch [JIT:0x8023a1e8->ppc:0x80630130=JIT:0xfe5a77db, ]
Block 0x80630000: applying patch [JIT:0x8026f66a->ppc:0x8063016c=JIT:0xfe5a79d9, ]
Block 0x80630000: applying patch [JIT:0x8026ec4a->ppc:0x80630158=JIT:0xfe5a7942, ]
Block 0x80630000: applying patch [JIT:0x8021feda->ppc:0x80630158=JIT:0xfe5a7942, ]
Block 0x80630000: applying patch [JIT:0x8021aa6a->ppc:0x8063012c=JIT:0xfe5a7789, ]
Block 0x80630000: applying patch [JIT:0x804440ba->ppc:0x80630120=JIT:0xfe5a76ff, ]
Block 0x80630000: applying patch [JIT:0x8026e07a->ppc:0x806300fc=JIT:0xfe5a7625, ]
Block 0x80630000: applying patch [JIT:0x80231f80->ppc:0x806301c8=JIT:0xfe5a7c8f, ]
Block 0x80630000: applying patch [JIT:0x8023a0b8->ppc:0x806301cc=JIT:0xfe5a7c94, ]
Block 0x80630000: applying patch [JIT:0x80216f70->ppc:0x806301a0=JIT:0xfe5a7b25, ]
Block 0x80630000: applying patch [JIT:0x8021226a->ppc:0x8063016c=JIT:0xfe5a79d9, ]
Block 0x80630000: applying patch [JIT:0x8023ccd2->ppc:0x806301c8=JIT:0xfe5a7c8f, ]
Block 0x80630000: applying patch [JIT:0x8025b882->ppc:0x806301c8=JIT:0xfe5a7c8f, ]
Block 0x80630000: applying patch [JIT:0x8026c3ca->ppc:0x8063019c=JIT:0xfe5a7ad3, ]
Block 0x80630000: applying patch [JIT:0x8021b10a->ppc:0x80630190=JIT:0xfe5a7a49, ]
Block 0x80630000: applying patch [JIT:0x8022292a->ppc:0x80630288=JIT:0xfe140263, ]
Block 0x80630000: applying patch [JIT:0x8021bdba->ppc:0x806301f8=JIT:0xfe5a7da6, ]
Block 0x80630000: applying patch [JIT:0x8023dcb2->ppc:0x80630200=JIT:0xfe5a7db6, ]
Block 0x80630000: applying patch [JIT:0x8023aef8->ppc:0x80630204=JIT:0xfe5a7dbe, ]
Block 0x80630000: applying patch [JIT:0x80227b5a->ppc:0x80630430=JIT:0xfe140dac, ]
Block 0x80630000: applying patch [JIT:0x80265e8a->ppc:0x80630274=JIT:0xfe1401d4, ]
Block 0x80630000: applying patch [JIT:0x802507b2->ppc:0x8063026c=JIT:0xfe1401c7, ]
Block 0x80630000: applying patch [JIT:0x802587a2->ppc:0x80630430=JIT:0xfe140dac, ]
Block 0x80630000: applying patch [JIT:0x805d1648->ppc:0x80630430=JIT:0xfe140dac, ]
Block 0x80630000: applying patch [JIT:0x802538a2->ppc:0x80630288=JIT:0xfe140263, ]
Block 0x80630000: applying patch [JIT:0x80240988->ppc:0x80630430=JIT:0xfe140dac, ]
Block 0x80630000: applying patch [JIT:0x80220f1a->ppc:0x80630424=JIT:0xfe140d86, ]
Block 0x80630000: applying patch [JIT:0x8022bcba->ppc:0x80630360=JIT:0xfe140810, ]
Block 0x80630000: applying patch [JIT:0x8021f61a->ppc:0x806303cc=JIT:0xfe140b4f, ]
Block 0x80630000: applying patch [JIT:0x8021f7ca->ppc:0x806303b0=JIT:0xfe140aaa, ]
Block 0x80630000: applying patch [JIT:0x8026b2ea->ppc:0x80630350=JIT:0xfe1407d5, ]
Block 0x80630000: applying patch [JIT:0x80251482->ppc:0x80630350=JIT:0xfe1407d5, ]
Block 0x80630000: applying patch [JIT:0x80304418->ppc:0x80630328=JIT:0xfe14066d, ]
Block 0x80630000: applying patch [JIT:0x801fedf2->ppc:0x806303c8=JIT:0xfe140b47, ]
Block 0x80630000: applying patch [JIT:0x8026360a->ppc:0x80630350=JIT:0xfe1407d5, ]
Block 0x80630000: applying patch [JIT:0x8020f0da->ppc:0x80630350=JIT:0xfe1407d5, ]
Block 0x80630000: applying patch [JIT:0x8023222a->ppc:0x80630324=JIT:0xfe14061b, ]
Block 0x80630000: applying patch [JIT:0x80242d62->ppc:0x80630318=JIT:0xfe140591, ]
Block 0x80630000: applying patch [JIT:0x8022a82a->ppc:0x806302e4=JIT:0xfe1404a1, ]
Block 0x80630000: applying patch [JIT:0x80220770->ppc:0x806303b0=JIT:0xfe140aaa, ]
Block 0x80630000: applying patch [JIT:0x80219670->ppc:0x80630388=JIT:0xfe140942, ]
Block 0x80630000: applying patch [JIT:0x8021117a->ppc:0x806303c8=JIT:0xfe140b47, ]
Block 0x80630000: applying patch [JIT:0x80262d9a->ppc:0x806303b0=JIT:0xfe140aaa, ]
Block 0x80630000: applying patch [JIT:0x802119ea->ppc:0x806303b0=JIT:0xfe140aaa, ]
Block 0x80630000: applying patch [JIT:0x80ae8502->ppc:0x80630384=JIT:0xfe1408f0, ]
Block 0x80630000: applying patch [JIT:0x8022cd5a->ppc:0x80630378=JIT:0xfe140866, ]
Block 0x80630000: applying patch [JIT:0x802646ea->ppc:0x806303c4=JIT:0xfe140b42, ]
Block 0x80630000: applying patch [JIT:0x8021de30->ppc:0x806303c8=JIT:0xfe140b47, ]
Block 0x80630000: applying patch [JIT:0x80214c2a->ppc:0x80630424=JIT:0xfe140d86, ]
Block 0x80630000: applying patch [JIT:0x8026db6a->ppc:0x80630424=JIT:0xfe140d86, ]
Block 0x80630000: applying patch [JIT:0x8026f15a->ppc:0x80630488=JIT:0xfe140f76, ]
Block 0x80630000: applying patch [JIT:0x8023a448->ppc:0x806304c4=JIT:0xfe14115f, ]
Block 0x80630000: applying patch [JIT:0x8023a7d8->ppc:0x806304c8=JIT:0xfe141164, ]
Block 0x80630000: applying patch [JIT:0x8023a6a8->ppc:0x806304b0=JIT:0xfe1410a9, ]
Block 0x80630000: applying patch [JIT:0x8022c85a->ppc:0x80630480=JIT:0xfe140f69, ]
Block 0x80630000: applying patch [JIT:0x80211d4a->ppc:0x806304c4=JIT:0xfe14115f, ]
Block 0x80630000: applying patch [JIT:0x8022d35a->ppc:0x806304a0=JIT:0xfe140fcd, ]
Block 0x80630000: applying patch [JIT:0x80215e1a->ppc:0x806305fc=JIT:0xfe14197a, ]
Block 0x80630000: applying patch [JIT:0x80246a42->ppc:0x80630588=JIT:0xfe14162e, ]
Block 0x80630000: applying patch [JIT:0x8026396a->ppc:0x806305fc=JIT:0xfe14197a, ]
Block 0x80630000: applying patch [JIT:0x80ae350a->ppc:0x806305d8=JIT:0xfe1418c8, ]
Block 0x80630000: applying patch [JIT:0x802427b2->ppc:0x80630578=JIT:0xfe1415f3, ]
Block 0x80630000: applying patch [JIT:0x8022562a->ppc:0x80630578=JIT:0xfe1415f3, ]
Block 0x80630000: applying patch [JIT:0x80239d28->ppc:0x80630550=JIT:0xfe14148e, ]
Block 0x80630000: applying patch [JIT:0x8026e3da->ppc:0x806305ec=JIT:0xfe141960, ]
Block 0x80630000: applying patch [JIT:0x802220ea->ppc:0x80630578=JIT:0xfe1415f3, ]
Block 0x80630000: applying patch [JIT:0x8022d76a->ppc:0x80630578=JIT:0xfe1415f3, ]
Block 0x80630000: applying patch [JIT:0x8098d022->ppc:0x8063054c=JIT:0xfe14143c, ]
Block 0x80630000: applying patch [JIT:0x802281da->ppc:0x80630540=JIT:0xfe1413b2, ]
Block 0x80630000: applying patch [JIT:0x8023243a->ppc:0x8063050c=JIT:0xfe1412c2, ]
Block 0x80630000: applying patch [JIT:0x8028a718->ppc:0x806305d8=JIT:0xfe1418c8, ]
Block 0x80630000: applying patch [JIT:0x80227020->ppc:0x806305b0=JIT:0xfe141760, ]
Block 0x80630000: applying patch [JIT:0x8026c21a->ppc:0x806305ec=JIT:0xfe141960, ]
Block 0x80630000: applying patch [JIT:0x80ae7ff2->ppc:0x806305d8=JIT:0xfe1418c8, ]
Block 0x80630000: applying patch [JIT:0x801ff662->ppc:0x806305d8=JIT:0xfe1418c8, ]
Block 0x80630000: applying patch [JIT:0x8022feca->ppc:0x806305ac=JIT:0xfe14170e, ]
Block 0x80630000: applying patch [JIT:0x80257392->ppc:0x806305a0=JIT:0xfe141684, ]
Block 0x80630000: applying patch [JIT:0x8098d2c2->ppc:0x806305f4=JIT:0xfe14196d, ]
Block 0x80630000: applying patch [JIT:0x80241368->ppc:0x806305f8=JIT:0xfe141972, ]
Block 0x80630000: applying patch [JIT:0x80226de0->ppc:0x806305fc=JIT:0xfe14197a, ]
Block 0x80630000: applying patch [JIT:0x80442aca->ppc:0x80630804=JIT:0xfe1427a3, ]
Block 0x80630000: applying patch [JIT:0x0->ppc:0x00000000=JIT:0x828927c8, ]

Program received signal SIGSEGV, Segmentation fault.
0x00441523 in ppc32_jit_tcb_apply_patches (cpu=0x801a8f18, block=0x80435c90, iop=0x8098d4a0)
    at /cygdrive/d/dev/hg/dynamips-community/stable/ppc32_jit.c:535
535              ppc32_jit_tcb_set_patch(jit_ptr,jit_dst);
(gdb) print *block
$4 = {start_ia = 2153971712, jit_insn_ptr = 0x825e07c8, acc_count = 79, ppc_code = 0xfa1a0000,
  ppc_trans_pos = 1024, jit_chunk_pos = 1, jit_ptr = 0xfe145f53 "\001", jit_buffer = 0x801ab2c0,
  jit_chunks = {0x0 <repeats 64 times>}, patch_table = 0x0, prev = 0x0, next = 0x8045be80,
  phys_page = 1584, phys_hash = 8263, phys_pprev = 0x0, phys_next = 0x0, target_bitmap = {0, 0, 0, 0, 0, 0,
    0, 0, 256, 0 <repeats 23 times>}, target_undef_cnt = 16}
(gdb) print *(struct ppc32_jit_patch_table *)0x82971c90
$5 = {next = 0x0, patches = {{next = 0x0, jit_insn = 0x80241ea2 "\017\204", ppc_ia = 2153972332}, {
      next = 0x0, jit_insn = 0x8021860a "\017\205", ppc_ia = 2153972352}, {next = 0x0,
      jit_insn = 0x8023ce82 "\017\204", ppc_ia = 2153972772}, {next = 0x0,
      jit_insn = 0x80245cb2 "\017\205", ppc_ia = 2153972772}, {next = 0x0,
      jit_insn = 0x8020eb5a "\017\204", ppc_ia = 2153971936}, {next = 0x0,
      jit_insn = 0x8022127a "\017\205", ppc_ia = 2153971848}, {next = 0x0,
      jit_insn = 0x80230f3a "\017\204", ppc_ia = 2153972168}, {next = 0x0,
      jit_insn = 0x80442fda "\017\205", ppc_ia = 2153972168}, {next = 0x0,
      jit_insn = 0x802473f2 "\017\205", ppc_ia = 2153972168}, {next = 0x0,
      jit_insn = 0x802632aa "\017\205", ppc_ia = 2153972076}, {next = 0x0,
      jit_insn = 0x8026d14a "\017\205", ppc_ia = 2153972168}, {next = 0x0,
      jit_insn = 0x802114da "\017\204", ppc_ia = 2153971896}, {next = 0x0,
      jit_insn = 0x80218c70 <incomplete sequence \351>, ppc_ia = 2153972168}, {next = 0x0,
      jit_insn = 0x8026b49a "\017\204", ppc_ia = 2153972084}, {next = 0x0,
      jit_insn = 0x8026345a "\017\204", ppc_ia = 2153972168}, {next = 0x0,
      jit_insn = 0x8023a1e8 <incomplete sequence \351>, ppc_ia = 2153972016}, {next = 0x0,
      jit_insn = 0x8026f66a "\017\205", ppc_ia = 2153972076}, {next = 0x0,
      jit_insn = 0x8026ec4a "\017\205", ppc_ia = 2153972056}, {next = 0x0,
      jit_insn = 0x8021feda "\017\205", ppc_ia = 2153972056}, {next = 0x0,
      jit_insn = 0x8021aa6a "\017\204", ppc_ia = 2153972012}, {next = 0x0,
      jit_insn = 0x804440ba "\017\204", ppc_ia = 2153972000}, {next = 0x0,
      jit_insn = 0x8026e07a "\017\205", ppc_ia = 2153971964}, {next = 0x0,
      jit_insn = 0x80231f80 <incomplete sequence \351>, ppc_ia = 2153972168}, {next = 0x0,
      jit_insn = 0x8023a0b8 <incomplete sequence \351>, ppc_ia = 2153972172}, {next = 0x0,
      jit_insn = 0x80216f70 <incomplete sequence \351>, ppc_ia = 2153972128}, {next = 0x0,
      jit_insn = 0x8021226a "\017\205", ppc_ia = 2153972076}, {next = 0x0,
      jit_insn = 0x8023ccd2 "\017\205", ppc_ia = 2153972168}, {next = 0x0,
      jit_insn = 0x8025b882 "\017\205", ppc_ia = 2153972168}, {next = 0x0,
      jit_insn = 0x8026c3ca "\017\204", ppc_ia = 2153972124}, {next = 0x0,
      jit_insn = 0x8021b10a "\017\204", ppc_ia = 2153972112}, {next = 0x0,
      jit_insn = 0x8022292a "\017\205", ppc_ia = 2153972360}, {next = 0x0,
      jit_insn = 0x8021bdba "\017\205", ppc_ia = 2153972216}}, cur_patch = 32}
(gdb) print *(struct ppc32_jit_patch_table *)0x82975d68
$6 = {next = 0x82971c90, patches = {{next = 0x0, jit_insn = 0x8023dcb2 "\017\204", ppc_ia = 2153972224}, {
      next = 0x0, jit_insn = 0x8023aef8 <incomplete sequence \351>, ppc_ia = 2153972228}, {next = 0x0,
      jit_insn = 0x80227b5a "\017\204", ppc_ia = 2153972784}, {next = 0x0,
      jit_insn = 0x80265e8a "\017\205", ppc_ia = 2153972340}, {next = 0x0,
      jit_insn = 0x802507b2 "\017\205", ppc_ia = 2153972332}, {next = 0x0,
      jit_insn = 0x802587a2 "\017\205", ppc_ia = 2153972784}, {next = 0x0,
      jit_insn = 0x805d1648 <incomplete sequence \351>, ppc_ia = 2153972784}, {next = 0x0,
      jit_insn = 0x802538a2 "\017\204", ppc_ia = 2153972360}, {next = 0x0,
      jit_insn = 0x80240988 <incomplete sequence \351>, ppc_ia = 2153972784}, {next = 0x0,
      jit_insn = 0x80220f1a "\017\205", ppc_ia = 2153972772}, {next = 0x0,
      jit_insn = 0x8022bcba "\017\205", ppc_ia = 2153972576}, {next = 0x0,
      jit_insn = 0x8021f61a "\017\205", ppc_ia = 2153972684}, {next = 0x0,
      jit_insn = 0x8021f7ca "\017\204", ppc_ia = 2153972656}, {next = 0x0,
      jit_insn = 0x8026b2ea "\017\205", ppc_ia = 2153972560}, {next = 0x0,
      jit_insn = 0x80251482 "\017\205", ppc_ia = 2153972560}, {next = 0x0,
      jit_insn = 0x80304418 <incomplete sequence \351>, ppc_ia = 2153972520}, {next = 0x0,
      jit_insn = 0x801fedf2 "\017\205", ppc_ia = 2153972680}, {next = 0x0,
      jit_insn = 0x8026360a "\017\205", ppc_ia = 2153972560}, {next = 0x0,
      jit_insn = 0x8020f0da "\017\205", ppc_ia = 2153972560}, {next = 0x0,
      jit_insn = 0x8023222a "\017\204", ppc_ia = 2153972516}, {next = 0x0,
      jit_insn = 0x80242d62 "\017\204", ppc_ia = 2153972504}, {next = 0x0,
      jit_insn = 0x8022a82a "\017\205", ppc_ia = 2153972452}, {next = 0x0,
      jit_insn = 0x80220770 <incomplete sequence \351>, ppc_ia = 2153972656}, {next = 0x0,
      jit_insn = 0x80219670 <incomplete sequence \351>, ppc_ia = 2153972616}, {next = 0x0,
      jit_insn = 0x8021117a "\017\205", ppc_ia = 2153972680}, {next = 0x0,
      jit_insn = 0x80262d9a "\017\205", ppc_ia = 2153972656}, {next = 0x0,
      jit_insn = 0x802119ea "\017\205", ppc_ia = 2153972656}, {next = 0x0,
      jit_insn = 0x80ae8502 "\017\204", ppc_ia = 2153972612}, {next = 0x0,
      jit_insn = 0x8022cd5a "\017\204", ppc_ia = 2153972600}, {next = 0x0,
      jit_insn = 0x802646ea "\017\205", ppc_ia = 2153972676}, {next = 0x0,
      jit_insn = 0x8021de30 <incomplete sequence \351>, ppc_ia = 2153972680}, {next = 0x0,
      jit_insn = 0x80214c2a "\017\205", ppc_ia = 2153972772}}, cur_patch = 32}
(gdb) print *(struct ppc32_jit_patch_table *)0x8297f9e8
$7 = {next = 0x82975d68, patches = {{next = 0x0, jit_insn = 0x8026db6a "\017\205", ppc_ia = 2153972772}, {
      next = 0x0, jit_insn = 0x8026f15a "\017\204", ppc_ia = 2153972872}, {next = 0x0,
      jit_insn = 0x8023a448 <incomplete sequence \351>, ppc_ia = 2153972932}, {next = 0x0,
      jit_insn = 0x8023a7d8 <incomplete sequence \351>, ppc_ia = 2153972936}, {next = 0x0,
      jit_insn = 0x8023a6a8 <incomplete sequence \351>, ppc_ia = 2153972912}, {next = 0x0,
      jit_insn = 0x8022c85a "\017\205", ppc_ia = 2153972864}, {next = 0x0,
      jit_insn = 0x80211d4a "\017\205", ppc_ia = 2153972932}, {next = 0x0,
      jit_insn = 0x8022d35a "\017\204", ppc_ia = 2153972896}, {next = 0x0,
      jit_insn = 0x80215e1a "\017\204", ppc_ia = 2153973244}, {next = 0x0,
      jit_insn = 0x80246a42 "\017\205", ppc_ia = 2153973128}, {next = 0x0,
      jit_insn = 0x8026396a "\017\205", ppc_ia = 2153973244}, {next = 0x0,
      jit_insn = 0x80ae350a "\017\204", ppc_ia = 2153973208}, {next = 0x0,
      jit_insn = 0x802427b2 "\017\205", ppc_ia = 2153973112}, {next = 0x0,
      jit_insn = 0x8022562a "\017\205", ppc_ia = 2153973112}, {next = 0x0,
      jit_insn = 0x80239d28 <incomplete sequence \351>, ppc_ia = 2153973072}, {next = 0x0,
      jit_insn = 0x8026e3da "\017\205", ppc_ia = 2153973228}, {next = 0x0,
      jit_insn = 0x802220ea "\017\205", ppc_ia = 2153973112}, {next = 0x0,
      jit_insn = 0x8022d76a "\017\205", ppc_ia = 2153973112}, {next = 0x0,
      jit_insn = 0x8098d022 "\017\204", ppc_ia = 2153973068}, {next = 0x0,
      jit_insn = 0x802281da "\017\204", ppc_ia = 2153973056}, {next = 0x0,
      jit_insn = 0x8023243a "\017\205", ppc_ia = 2153973004}, {next = 0x0,
      jit_insn = 0x8028a718 <incomplete sequence \351>, ppc_ia = 2153973208}, {next = 0x0,
      jit_insn = 0x80227020 <incomplete sequence \351>, ppc_ia = 2153973168}, {next = 0x0,
      jit_insn = 0x8026c21a "\017\205", ppc_ia = 2153973228}, {next = 0x0,
      jit_insn = 0x80ae7ff2 "\017\205", ppc_ia = 2153973208}, {next = 0x0,
      jit_insn = 0x801ff662 "\017\205", ppc_ia = 2153973208}, {next = 0x0,
      jit_insn = 0x8022feca "\017\204", ppc_ia = 2153973164}, {next = 0x0,
      jit_insn = 0x80257392 "\017\204", ppc_ia = 2153973152}, {next = 0x0,
      jit_insn = 0x8098d2c2 "\017\205", ppc_ia = 2153973236}, {next = 0x0,
      jit_insn = 0x80241368 <incomplete sequence \351>, ppc_ia = 2153973240}, {next = 0x0,
      jit_insn = 0x80226de0 <incomplete sequence \351>, ppc_ia = 2153973244}, {next = 0x0,
      jit_insn = 0x80442aca "\017\205", ppc_ia = 2153973764}}, cur_patch = 32}
(gdb) print *(struct ppc32_jit_patch_table *)0x82987d68
$8 = {next = 0x82991bc8, patches = {{next = 0x82995988, jit_insn = 0x0, ppc_ia = 0}, {next = 0x0,
      jit_insn = 0x1 <Address 0x1 out of bounds>, ppc_ia = 2153973716}, {next = 0x0,
      jit_insn = 0x805c7900 <incomplete sequence \351>, ppc_ia = 2153973768}, {next = 0x0,
      jit_insn = 0x80263e7a "\017\204", ppc_ia = 2153973404}, {next = 0x0,
      jit_insn = 0x8023d4c2 "\017\205", ppc_ia = 2153973636}, {next = 0x0,
      jit_insn = 0x8022255a "\017\205", ppc_ia = 2153973636}, {next = 0x0,
      jit_insn = 0x802467d2 "\017\205", ppc_ia = 2153973544}, {next = 0x0,
      jit_insn = 0x8026d4aa "\017\205", ppc_ia = 2153973636}, {next = 0x0,
      jit_insn = 0x8024c1a2 "\017\204", ppc_ia = 2153973364}, {next = 0x0,
      jit_insn = 0x80227450 <incomplete sequence \351>, ppc_ia = 2153973636}, {next = 0x0,
      jit_insn = 0x8022189a "\017\204", ppc_ia = 2153973552}, {next = 0x0,
      jit_insn = 0x8026beba "\017\204", ppc_ia = 2153973636}, {next = 0x0,
      jit_insn = 0x80229220 <incomplete sequence \351>, ppc_ia = 2153973484}, {next = 0x0,
      jit_insn = 0x8024e4c2 "\017\205", ppc_ia = 2153973544}, {next = 0x0,
      jit_insn = 0x8023edc2 "\017\205", ppc_ia = 2153973524}, {next = 0x0,
      jit_insn = 0x8021d6da "\017\205", ppc_ia = 2153973524}, {next = 0x0,
      jit_insn = 0x80246082 "\017\204", ppc_ia = 2153973480}, {next = 0x0,
      jit_insn = 0x80252442 "\017\204", ppc_ia = 2153973468}, {next = 0x0,
      jit_insn = 0x80442e2a "\017\205", ppc_ia = 2153973432}, {next = 0x0,
      jit_insn = 0x80289a08 <incomplete sequence \351>, ppc_ia = 2153973636}, {next = 0x0,
      jit_insn = 0x8022e6f0 <incomplete sequence \351>, ppc_ia = 2153973640}, {next = 0x0,
      jit_insn = 0x805d3058 <incomplete sequence \351>, ppc_ia = 2153973596}, {next = 0x0,
      jit_insn = 0x80252652 "\017\205", ppc_ia = 2153973544}, {next = 0x0,
      jit_insn = 0x8021b32a "\017\205", ppc_ia = 2153973636}, {next = 0x0,
      jit_insn = 0x802666fa "\017\205", ppc_ia = 2153973636}, {next = 0x0,
      jit_insn = 0x80267b3a "\017\204", ppc_ia = 2153973592}, {next = 0x0,
      jit_insn = 0x80242602 "\017\204", ppc_ia = 2153973580}, {next = 0x0,
      jit_insn = 0x802657ca "\017\205", ppc_ia = 2153973712}, {next = 0x0,
      jit_insn = 0x8026603a "\017\205", ppc_ia = 2153973764}, {next = 0x0,
      jit_insn = 0x8021246a "\017\205", ppc_ia = 2153973764}, {next = 0x0,
      jit_insn = 0x80243c12 "\017\204", ppc_ia = 2153973768}, {next = 0x0,
      jit_insn = 0x8026aa7a "\017\205", ppc_ia = 2153973764}}, cur_patch = 32}
(gdb) print *(struct ppc32_jit_patch_table *)0x82990e50
$9 = {next = 0x82990e48, patches = {{next = 0x82990e48, jit_insn = 0x0, ppc_ia = 0}, {
      next = 0x6123e6fc <_gm_+316>, jit_insn = 0x4 <Address 0x4 out of bounds>, ppc_ia = 2153973768}, {
      next = 0x0, jit_insn = 0x801d8442 <incomplete sequence \351>, ppc_ia = 2153974024}, {next = 0x0,
      jit_insn = 0x8020f4aa "\017\204", ppc_ia = 2153973848}, {next = 0x0,
      jit_insn = 0x802197aa "\017\205", ppc_ia = 2153974000}, {next = 0x0,
      jit_insn = 0x80249122 "\017\205", ppc_ia = 2153974000}, {next = 0x0,
      jit_insn = 0x8023b8e2 "\017\205", ppc_ia = 2153973924}, {next = 0x0,
      jit_insn = 0x80444a4a "\017\205", ppc_ia = 2153974000}, {next = 0x0,
      jit_insn = 0x8023d668 <incomplete sequence \351>, ppc_ia = 2153974004}, {next = 0x0,
      jit_insn = 0x80252972 "\017\205", ppc_ia = 2153974000}, {next = 0x0,
      jit_insn = 0x8023f122 "\017\204", ppc_ia = 2153973964}, {next = 0x0,
      jit_insn = 0x8023b7a8 <incomplete sequence \351>, ppc_ia = 2153973980}, {next = 0x0,
      jit_insn = 0x802248ba "\017\205", ppc_ia = 2153974000}, {next = 0x0,
      jit_insn = 0x8022c48a "\017\204", ppc_ia = 2153973812}, {next = 0x0,
      jit_insn = 0x802404b2 "\017\205", ppc_ia = 2153974448}, {next = 0x0,
      jit_insn = 0x8023e652 "\017\205", ppc_ia = 2153974080}, {next = 0x0,
      jit_insn = 0x802697ea "\017\204", ppc_ia = 2153974368}, {next = 0x0,
      jit_insn = 0x802234fa "\017\205", ppc_ia = 2153974284}, {next = 0x0,
      jit_insn = 0x8024ea22 "\017\205", ppc_ia = 2153974840}, {next = 0x0,
      jit_insn = 0x802105ba "\017\204", ppc_ia = 2153974836}, {next = 0x0,
      jit_insn = 0x8021e32a "\017\205", ppc_ia = 2153974268}, {next = 0x0,
      jit_insn = 0x80264daa "\017\205", ppc_ia = 2153974268}, {next = 0x0,
      jit_insn = 0x803e4668 <incomplete sequence \351>, ppc_ia = 2153974228}, {next = 0x0,
      jit_insn = 0x8023bf22 "\017\205", ppc_ia = 2153974748}, {next = 0x0,
      jit_insn = 0x802681fa "\017\205", ppc_ia = 2153974268}, {next = 0x0,
      jit_insn = 0x80243a12 "\017\205", ppc_ia = 2153974268}, {next = 0x0,
      jit_insn = 0x8022334a "\017\204", ppc_ia = 2153974224}, {next = 0x0,
      jit_insn = 0x80217baa "\017\204", ppc_ia = 2153974212}, {next = 0x0,
      jit_insn = 0x8026711a "\017\205", ppc_ia = 2153974160}, {next = 0x0,
      jit_insn = 0x80226a10 <incomplete sequence \351>, ppc_ia = 2153974836}, {next = 0x0,
      jit_insn = 0x8021dcb0 <incomplete sequence \351>, ppc_ia = 2153974324}, {next = 0x0,
      jit_insn = 0x8021039a "\017\205", ppc_ia = 2153974748}}, cur_patch = 32}
(gdb) print *(struct ppc32_jit_patch_table *)0x82990fe0
$10 = {next = 0x82990fd8, patches = {{next = 0x82990fd8, jit_insn = 0x0, ppc_ia = 0}, {
      next = 0x6123e6f8 <_gm_+312>, jit_insn = 0x3 <Address 0x3 out of bounds>, ppc_ia = 2153974836}, {
      next = 0x0, jit_insn = 0x80230b2a "\017\204", ppc_ia = 2153974320}, {next = 0x0,
      jit_insn = 0x80262a3a "\017\204", ppc_ia = 2153974308}, {next = 0x0,
      jit_insn = 0x80256718 <incomplete sequence \351>, ppc_ia = 2153974836}, {next = 0x0,
      jit_insn = 0x8021630a "\017\205", ppc_ia = 2153974836}, {next = 0x0,
      jit_insn = 0x80224be0 <incomplete sequence \351>, ppc_ia = 2153974424}, {next = 0x0,
      jit_insn = 0x80245902 "\017\205", ppc_ia = 2153974748}, {next = 0x0,
      jit_insn = 0x802637ba "\017\205", ppc_ia = 2153974836}, {next = 0x0,
      jit_insn = 0x80ae31aa "\017\204", ppc_ia = 2153974408}, {next = 0x0,
      jit_insn = 0x802160a0 <incomplete sequence \351>, ppc_ia = 2153974836}, {next = 0x0,
      jit_insn = 0x802131ea "\017\204", ppc_ia = 2153974476}, {next = 0x0,
      jit_insn = 0x805beee8 <incomplete sequence \351>, ppc_ia = 2153974536}, {next = 0x0,
      jit_insn = 0x802315f0 <incomplete sequence \351>, ppc_ia = 2153974540}, {next = 0x0,
      jit_insn = 0x80229940 <incomplete sequence \351>, ppc_ia = 2153974516}, {next = 0x0,
      jit_insn = 0x80252be2 "\017\205", ppc_ia = 2153974468}, {next = 0x0,
      jit_insn = 0x8021d4ba "\017\205", ppc_ia = 2153974536}, {next = 0x0,
      jit_insn = 0x8026762a "\017\204", ppc_ia = 2153974500}, {next = 0x0,
      jit_insn = 0x8026855a "\017\204", ppc_ia = 2153974840}, {next = 0x0,
      jit_insn = 0x8023d7a2 "\017\205", ppc_ia = 2153974756}, {next = 0x0,
      jit_insn = 0x8021c94a "\017\205", ppc_ia = 2153974840}, {next = 0x0,
      jit_insn = 0x802589b2 "\017\204", ppc_ia = 2153974836}, {next = 0x0,
      jit_insn = 0x8021c72a "\017\205", ppc_ia = 2153974732}, {next = 0x0,
      jit_insn = 0x8024c9b2 "\017\205", ppc_ia = 2153974732}, {next = 0x0,
      jit_insn = 0x8023a908 <incomplete sequence \351>, ppc_ia = 2153974692}, {next = 0x0,
      jit_insn = 0x8026b9aa "\017\205", ppc_ia = 2153974748}, {next = 0x0,
      jit_insn = 0x80216cea "\017\205", ppc_ia = 2153974732}, {next = 0x0,
      jit_insn = 0x8026deca "\017\205", ppc_ia = 2153974732}, {next = 0x0,
      jit_insn = 0x8026546a "\017\204", ppc_ia = 2153974688}, {next = 0x0,
      jit_insn = 0x8022b4ea "\017\204", ppc_ia = 2153974676}, {next = 0x0,
      jit_insn = 0x80244012 "\017\205", ppc_ia = 2153974624}, {next = 0x0,
      jit_insn = 0x80240858 <incomplete sequence \351>, ppc_ia = 2153974836}}, cur_patch = 32}
(gdb) print *(struct ppc32_jit_patch_table *)0x82991170
$11 = {next = 0x82991bc8, patches = {{next = 0x82995988, jit_insn = 0x0, ppc_ia = 0}, {next = 0x0,
      jit_insn = 0x1 <Address 0x1 out of bounds>, ppc_ia = 2153974796}, {next = 0x0,
      jit_insn = 0x8021506a "\017\205", ppc_ia = 2153974748}, {next = 0x0,
      jit_insn = 0x8024a372 "\017\205", ppc_ia = 2153974836}, {next = 0x0,
      jit_insn = 0x80240182 "\017\205", ppc_ia = 2153974836}, {next = 0x0,
      jit_insn = 0x8026cc3a "\017\204", ppc_ia = 2153974792}, {next = 0x0,
      jit_insn = 0x8021168a "\017\204", ppc_ia = 2153974780}, {next = 0x0,
      jit_insn = 0x8024d302 "\017\204", ppc_ia = 2153974868}, {next = 0x0,
      jit_insn = 0x8023f2d2 "\017\204", ppc_ia = 2153975572}, {next = 0x0,
      jit_insn = 0x805d4b70 <incomplete sequence \351>, ppc_ia = 2153975624}, {next = 0x0,
      jit_insn = 0x80245ab2 "\017\205", ppc_ia = 2153975580}, {next = 0x0,
      jit_insn = 0x80259922 "\017\204", ppc_ia = 2153975004}, {next = 0x0,
      jit_insn = 0x8021a2ea "\017\205", ppc_ia = 2153974916}, {next = 0x0,
      jit_insn = 0x8022e48a "\017\204", ppc_ia = 2153975236}, {next = 0x0,
      jit_insn = 0x80212fea "\017\205", ppc_ia = 2153975236}, {next = 0x0,
      jit_insn = 0x8021c3ca "\017\205", ppc_ia = 2153975236}, {next = 0x0,
      jit_insn = 0x80219b7a "\017\205", ppc_ia = 2153975144}, {next = 0x0,
      jit_insn = 0x80242252 "\017\205", ppc_ia = 2153975236}, {next = 0x0,
      jit_insn = 0x80226c3a "\017\204", ppc_ia = 2153974964}, {next = 0x0,
      jit_insn = 0x8023bc38 <incomplete sequence \351>, ppc_ia = 2153975236}, {next = 0x0,
      jit_insn = 0x80257602 "\017\204", ppc_ia = 2153975152}, {next = 0x0,
      jit_insn = 0x802683aa "\017\204", ppc_ia = 2153975236}, {next = 0x0,
      jit_insn = 0x80219170 <incomplete sequence \351>, ppc_ia = 2153975084}, {next = 0x0,
      jit_insn = 0x8024e252 "\017\205", ppc_ia = 2153975144}, {next = 0x0,
      jit_insn = 0x8026963a "\017\205", ppc_ia = 2153975124}, {next = 0x0,
      jit_insn = 0x8022f98a "\017\205", ppc_ia = 2153975124}, {next = 0x0,
      jit_insn = 0x8024f8d2 "\017\204", ppc_ia = 2153975080}, {next = 0x0,
      jit_insn = 0x8021fd2a "\017\204", ppc_ia = 2153975068}, {next = 0x0,
      jit_insn = 0x80ae36ba "\017\205", ppc_ia = 2153975032}, {next = 0x0,
      jit_insn = 0x80240ea8 <incomplete sequence \351>, ppc_ia = 2153975236}, {next = 0x0,
      jit_insn = 0x80219f90 <incomplete sequence \351>, ppc_ia = 2153975240}, {next = 0x0,
      jit_insn = 0x80238ee8 <incomplete sequence \351>, ppc_ia = 2153975196}}, cur_patch = 32}
(gdb) print *(struct ppc32_jit_patch_table *)0x82991bd0
$12 = {next = 0x82995988, patches = {{next = 0x82987d60, jit_insn = 0x0, ppc_ia = 0}, {next = 0x0,
      jit_insn = 0x1 <Address 0x1 out of bounds>, ppc_ia = 2153975236}, {next = 0x0,
      jit_insn = 0x8026c8da "\017\205", ppc_ia = 2153975236}, {next = 0x0,
      jit_insn = 0x801fff72 "\017\204", ppc_ia = 2153975192}, {next = 0x0,
      jit_insn = 0x80a00dda "\017\204", ppc_ia = 2153975180}, {next = 0x0,
      jit_insn = 0x80250c32 "\017\204", ppc_ia = 2153975580}, {next = 0x0,
      jit_insn = 0x80212c3a "\017\205", ppc_ia = 2153975612}, {next = 0x0,
      jit_insn = 0x80443f0a "\017\205", ppc_ia = 2153975472}, {next = 0x0,
      jit_insn = 0x8020f28a "\017\205", ppc_ia = 2153975556}, {next = 0x0,
      jit_insn = 0x8022dc4a "\017\204", ppc_ia = 2153975552}, {next = 0x0,
      jit_insn = 0x802194ca "\017\205", ppc_ia = 2153975448}, {next = 0x0,
      jit_insn = 0x8023e1c2 "\017\205", ppc_ia = 2153975448}, {next = 0x0,
      jit_insn = 0x80224ec0 <incomplete sequence \351>, ppc_ia = 2153975408}, {next = 0x0,
      jit_insn = 0x8021b80a "\017\205", ppc_ia = 2153975464}, {next = 0x0,
      jit_insn = 0x80259612 "\017\205", ppc_ia = 2153975448}, {next = 0x0,
      jit_insn = 0x80255cf2 "\017\205", ppc_ia = 2153975448}, {next = 0x0,
      jit_insn = 0x80250a22 "\017\204", ppc_ia = 2153975404}, {next = 0x0,
      jit_insn = 0x80269b4a "\017\204", ppc_ia = 2153975392}, {next = 0x0,
      jit_insn = 0x8026d9ba "\017\205", ppc_ia = 2153975340}, {next = 0x0,
      jit_insn = 0x805d4580 <incomplete sequence \351>, ppc_ia = 2153975552}, {next = 0x0,
      jit_insn = 0x80227690 <incomplete sequence \351>, ppc_ia = 2153975556}, {next = 0x0,
      jit_insn = 0x805d4f00 <incomplete sequence \351>, ppc_ia = 2153975512}, {next = 0x0,
      jit_insn = 0x8026438a "\017\205", ppc_ia = 2153975464}, {next = 0x0,
      jit_insn = 0x802272aa "\017\205", ppc_ia = 2153975552}, {next = 0x0,
      jit_insn = 0x80259d42 "\017\205", ppc_ia = 2153975552}, {next = 0x0,
      jit_insn = 0x80267e9a "\017\204", ppc_ia = 2153975508}, {next = 0x0,
      jit_insn = 0x8026a8ca "\017\204", ppc_ia = 2153975496}, {next = 0x0,
      jit_insn = 0x80263cca "\017\204", ppc_ia = 2153975580}, {next = 0x0,
      jit_insn = 0x8021091a "\017\205", ppc_ia = 2153974856}, {next = 0x0,
      jit_insn = 0x80240fd8 <incomplete sequence \351>, ppc_ia = 2153975624}, {next = 0x0,
      jit_insn = 0x80ae81a2 "\017\205", ppc_ia = 2153975620}, {next = 0x0,
      jit_insn = 0x80224ffa "\017\204", ppc_ia = 2153975612}}, cur_patch = 32}
(gdb) print *(struct ppc32_jit_patch_table *)0x82995990
$13 = {next = 0x82987d60, patches = {{next = 0x82991bc8, jit_insn = 0x0, ppc_ia = 0}, {
      next = 0x6123e6f0 <_gm_+304>, jit_insn = 0x1 <Address 0x1 out of bounds>, ppc_ia = 2153975624}, {
      next = 0x0, jit_insn = 0x8021e6fa "\017\205", ppc_ia = 2153975700}, {next = 0x0, jit_insn = 0x0,
      ppc_ia = 0} <repeats 29 times>}, cur_patch = 3}

Notice that from $8 to $13 all of them have unexpected next values and the first two items of the patches array are corrupted.
Ok, it seems the memory got corrupted without crashing immediately.

Current suspicion is a buffer underflow or buffer overflow. (evil bastards, very hard to find)
Next I'm gonna try using a memory manager to check how much memory got affected.
On the back burner, there's also the possibility that it's related to being multi-threaded.
Top
 Profile  
 
grossmj
 Post subject: Re: dynamips crash, compiled in cygwin with x86 arch
PostPosted: Wed Jun 05, 2013 3:53 am 
Offline
Site Admin

Joined: Sat Oct 11, 2008 1:41 pm
Posts: 2668
Location: Canada
I see you are going deep there. I would love to help but the actual emulation code of Dynamips is a mystery to me. I still understand your debugging process but that's it.

Hopefully you can fix the problem :) It would be awesome.

_________________
Jeremy, GNS3 Programmer & Benevolent Dictator for Life.
Top
 Profile  
 
flaviojs
 Post subject: Re: dynamips crash, compiled in cygwin with x86 arch
PostPosted: Wed Jun 05, 2013 3:14 pm 
Offline

Joined: Wed May 22, 2013 7:48 am
Posts: 93
Location: Portugal
I added memwatch.
I found out that it's actually freed data and no buffer underflow or overflow was detected.
Code:
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 13928.0x145c]
0x004421bf in ppc32_jit_tcb_apply_patches (cpu=0x8c1ef0c0, block=0x8c35a598, iop=0x8c29e978)
    at /cygdrive/d/dev/hg/dynamips-community/stable/ppc32_jit.c:540
540              ppc32_jit_tcb_set_patch(jit_ptr,jit_dst);
(gdb) print *patch
$1 = {next = 0xfdfdfdfd, jit_insn = 0xfdfdfdfd "P", ppc_ia = 4261281277}

Ok, now we know it's actually a dangling pointer.

Suspecting that I might have missed a path to ppc32_jit_tcb_free_patches before the patches are applied, I rigged the code to confirm it.
Code:
...
ppc32_jit_tcb_record_patch - start pthread_self()=-2145329424
Block 0x8029b000: recording patch [JIT:0x80303a4a->ppc:0x8029bff8], MTP=1019
ppc32_jit_tcb_record_patch - end pthread_self()=-2145329424
ppc32_op_gen_page - end section 'Generate JIT opcodes'
ppc32_op_gen_page - start section 'Generate JIT code for each instruction in page'
ppc32_jit_tcb_free - pthread_self()=-2145329424
ppc32_jit_tcb_free_patches - start pthread_self()=-2145329424
GCC_SHOULD_BREAK

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 8204.0x1954]
0x0044246b in ppc32_jit_tcb_free_patches (block=0x806d5590)
    at /cygdrive/d/dev/hg/dynamips-community/stable/ppc32_jit.c:596
596           p[0] = 0; // force a SIGSEGV
(gdb) bt full
#0  0x0044246b in ppc32_jit_tcb_free_patches (block=0x806d5590)
    at /cygdrive/d/dev/hg/dynamips-community/stable/ppc32_jit.c:596
        p = 0x0
        p = <optimized out>
        next = <optimized out>
#1  0x004428b1 in ppc32_jit_tcb_free (cpu=0x8020e2c8, block=0x806d5590, list_removal=1)
    at /cygdrive/d/dev/hg/dynamips-community/stable/ppc32_jit.c:709
No locals.
#2  0x00441629 in ppc32_jit_flush (cpu=0x8020e2c8, threshold=4294967295)
    at /cygdrive/d/dev/hg/dynamips-community/stable/ppc32_jit.c:172
        p = 0x806d5590
        next = 0x806d1538
        ia_hash = 127361
        count = 0
#3  0x004417ff in exec_page_alloc (cpu=0x8020e2c8)
    at /cygdrive/d/dev/hg/dynamips-community/stable/ppc32_jit.c:225
        p = 0x0
        count = <optimized out>
#4  0x0044255f in ppc32_jit_tcb_adjust_buffer (cpu=0x8020e2c8, block=0x806da6a8)
    at /cygdrive/d/dev/hg/dynamips-community/stable/ppc32_jit.c:629
        new_buffer = <optimized out>
#5  0x00443875 in ppc32_op_gen_page (cpu=0x8020e2c8, b=0x806da6a8)
    at /cygdrive/d/dev/hg/dynamips-community/stable/ppc32_jit.c:1166
        tag = <optimized out>
        gcpu = 0x8020e110
        iop = <optimized out>
        cur_ia = <optimized out>
        jit_ptr = 0xfda2fddd "\213_\034\213\303\301\350\f%\377?"
        i = 840
#6  0x00443a88 in ppc32_jit_tcb_compile (cpu=0x8020e2c8, vaddr=2150216988)
    at /cygdrive/d/dev/hg/dynamips-community/stable/ppc32_jit.c:1223
        block = 0x806da6a8
        page_addr = 2150215680
#7  0x0044404b in ppc32_jit_run_cpu (gen=0x8020e110)
    at /cygdrive/d/dev/hg/dynamips-community/stable/ppc32_jit.c:1370
        cpu = 0x8020e2c8
        timer_irq_thread = 0x8020df80
        block = 0x0
        ia_hash = 68432
        timer_irq_check = 892
#8  0x610fe08a in pthread::thread_init_wrapper(void*) () from /usr/bin/cygwin1.dll
No symbol table info available.
#9  0x610874d2 in thread_wrapper(void*) () from /usr/bin/cygwin1.dll
No symbol table info available.

There it is... then the problem should be a logic or design error in the recompile JIT code path.
I'm gonna be more careful here and take my time until I understand how this is meant to work.

Current hypothesis:
  • ppc32_jit_tcb_record_patch puts a pointer to an entry on the array of patches (of the struct ppc32_jit_patch_table, stored in b->patch_table) into the arg_ptr field of the op code structure.
  • ppc32_jit_tcb_adjust_buffer adjusts the JIT buffer and frees the patch tables (through exec_page_alloc and ppc32_jit_flush), leaving dangling pointers in the op codes.
  • ppc32_jit_tcb_apply_patches is run on an op code of type JIT_OP_INSN_OUTPUT that has the dangling pointer in the arg_ptr field
  • an invalid jit_ptr is produced from the bogus data and ppc32_jit_tcb_set_patch (x86_patch) crashes when reading the type of instruction from that address
At first glance a quick fix would be to mark the block as being recompiled and prevent it from being flushed.

-----

grossmj wrote:
I see you are going deep there. I would love to help but the actual emulation code of Dynamips is a mystery to me. I still understand your debugging process but that's it.

Hopefully you can fix the problem :) It would be awesome.

I'll keep going as long as I have a hypothesis to explore. ;D

Hmm, how about adding a nojit build of dynamips with GNS3 along with the current one? It eats up more cpu but hasn't crashed on me yet. (only tried it once, though)
Top
 Profile  
 
claydon_dan
 Post subject: Re: dynamips crash, compiled in cygwin with x86 arch
PostPosted: Thu Jun 06, 2013 9:07 am 
Offline

Joined: Sun Sep 16, 2012 9:55 pm
Posts: 541
Location: England
Quote:
Hmm, how about adding a nojit build of dynamips with GNS3 along with the current one? It eats up more cpu but hasn't crashed on me yet. (only tried it once, though)

I may be wrong but it would seem that is what the 'Enable JIT sharing support option would do (see attached)


Attachments:
JIT-Sharing.png
JIT-Sharing.png [ 66.57 KiB | Viewed 6133 times ]

_________________
Daniel
Forum Moderator & Debian Package Maintainer for GNS3, Dynamips & VPCS.
Standalone DEB Packages are available from http://gns3.serverb.co.uk - To be updated!
Top
 Profile  
 
flaviojs
 Post subject: Re: dynamips crash, compiled in cygwin with x86 arch
PostPosted: Thu Jun 06, 2013 2:39 pm 
Offline

Joined: Wed May 22, 2013 7:48 am
Posts: 93
Location: Portugal
Digging through GNS3+Dynagen I found that the option is using the dynamips command "vm set_tsg <object> <group>", which is only present in the unstable version.
I'm working in the stable version so it doesn't affect what i'm doing.

In stable each cpu (re)compiles the chunks of the image (into native code) separately, even if they share the exact same image.
That option probably enables the sharing of the (re)compiled code, so the other routers in the group reap the benefits immediately.

A nojit build does absolutely no (re)compiling, it just reads instructions and executes them with it's own functions.
Top
 Profile  
 
flaviojs
 Post subject: Re: dynamips crash, compiled in cygwin with x86 arch
PostPosted: Fri Jun 07, 2013 5:35 am 
Offline

Joined: Wed May 22, 2013 7:48 am
Posts: 93
Location: Portugal
flaviojs wrote:
Current hypothesis:
  • ppc32_jit_tcb_record_patch puts a pointer to an entry on the array of patches (of the struct ppc32_jit_patch_table, stored in b->patch_table) into the arg_ptr field of the op code structure.
  • ppc32_jit_tcb_adjust_buffer adjusts the JIT buffer and frees the patch tables (through exec_page_alloc and ppc32_jit_flush), leaving dangling pointers in the op codes.
  • ppc32_jit_tcb_apply_patches is run on an op code of type JIT_OP_INSN_OUTPUT that has the dangling pointer in the arg_ptr field
  • an invalid jit_ptr is produced from the bogus data and ppc32_jit_tcb_set_patch (x86_patch) crashes when reading the type of instruction from that address
At first glance a quick fix would be to mark the block as being recompiled and prevent it from being flushed.

I was testing if the quick fix worked by letting it run for 24h, but I ran out of memory after 10h. (longest run of the x86 arch with the scenario)

So... let's see if I added a memory leak (start dynamips, open topology in GNS3, start 2 routers of the scenario, wait 1 minute, stop routers, stop dynamips with a single Ctrl+C)
  • x86 arch with quick fix, memwatch detected 723k leaks. O.O
  • x86 arch, memwatch detected 722k leaks. >.>
  • nojit arch, memwatch detected 2k leaks. D=
  • r0, x86 arch, memwatch detected 724k leaks...
  • dynamips-0.2.8-RC2, x86 arch, memwatch detected 725k leaks. (╯°□°)╯︵ ┻━┻

Ok, looks the like memory leaks were a problem even before the community fork...
Anyway, looks like the proposed quick fix works. I'm still gonna take my time until I understand how this is meant to work.


Last edited by flaviojs on Mon Jun 10, 2013 9:31 pm, edited 1 time in total.

Top
 Profile  
 
flaviojs
 Post subject: Re: dynamips crash, compiled in cygwin with x86 arch
PostPosted: Mon Jun 10, 2013 9:27 pm 
Offline

Joined: Wed May 22, 2013 7:48 am
Posts: 93
Location: Portugal
Done, the routers that use the ppc32 cpu are C1700, C2600, and C7200 with NPE-G2 (c7200p).

The attached patch was generated with the command "hg export -r24 > patch.diff".


Attachments:
File comment: Avoid dangling pointers when the ppc32 cpu is recompiling a tcb and an exec page needs to be allocated, causing the tcb to be flushed.
patch.diff [3.02 KiB]
Downloaded 225 times


Last edited by flaviojs on Tue Jun 11, 2013 12:14 am, edited 2 times in total.
Top
 Profile  
 
grossmj
 Post subject: Re: dynamips crash, compiled in cygwin with x86 arch
PostPosted: Mon Jun 10, 2013 10:21 pm 
Offline
Site Admin

Joined: Sat Oct 11, 2008 1:41 pm
Posts: 2668
Location: Canada
Quote:
Done, the routers that use the ppc32 cpu are C1700, C2600, and C7200 with NPE-G2 (c7200p).


Wow, good job! I suppose you extensively tested your patch?

Thanks!



_________________
Jeremy, GNS3 Programmer & Benevolent Dictator for Life.
Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  Page 1 of 2
  [ 19 posts ]  Go to page 1, 2  Next

Board index » GNS3 » Development » Bug reports

All times are UTC


Who is online

Users browsing this forum: No registered users and 0 guests

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB® Forum Software © phpBB Group

phpBB SEO