Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

mythfrontend, mythtv-setup master: random segfaults on startup #593

Open
ulmus-scott opened this issue Jun 25, 2022 · 5 comments
Open

mythfrontend, mythtv-setup master: random segfaults on startup #593

ulmus-scott opened this issue Jun 25, 2022 · 5 comments

Comments

@ulmus-scott
Copy link
Contributor

  • Platform: Xubuntu 22.04 Linux scott-desktop 5.15.0-40-generic #43-Ubuntu SMP Wed Jun 15 12:54:21 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

  • MythTV version: master (attachments generated with MythEDID: fix out of bounds memory access #592 )

  • Package version: n/a (git)

  • Component: mythfrontend, mythtv-setup

What steps will reproduce the bug?

mythfrontend or mythtv-setup, no user input required.

How often does it reproduce? Is there a required condition?

It appears to randomly occur about 10% of the time (without having counted).

What is the expected behaviour?

No segmentation faults terminating the process.

What do you see instead?

The screen briefly flashes with the expected menu and then the program crashes.

Additional information

I have reproduced this on my other Xubuntu 22.04 test system which uses an Intel G3258's iGPU instead of an Nvidia GTX 970. The segfaults occured with either nouveau or the Nvidia binary driver.

Valgrind revealed #592 , but I didn't see anything else relevant mythfrontend_valgrind.txt . I have not tried recompiling with --compile-type=debug for a clearer valgrind output.

See attached logs and gdb backtraces.

Thread 1 (Thread 0x7f1bf3215640 (LWP 3607)):
#0  0x00007f1c3c3f2560 in  ()
#1  0x00007f1cf5a9db43 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
        ret = <optimized out>
        pd = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140723785565568, 2102220873009990209, 139758019892800, 0, 139762357360720, 140723785565920, -2081946173401462207, -2083620794559231423}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
#2  0x00007f1cf5b2fa00 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

I don't know what to make of the backtraces, in particular #0 0x00007f1c3c3f2560 in ().

mythfrontend_backtrace.3574.txt
mythfrontend_seg_fault.3574.log
mythtv-setup_backtrace.3973.txt
mythtv-setup_seg_fault.3973.log
@ulmus-scott
Copy link
Contributor Author

I started to attempt a bisect with the first commit with the DB version 1376, which still failed, but the backtrace suggests the segfault occurs in libCEC while it is being closed:
mythfrontend_seg_fault_on_bisect.log

Thread 30 "mythfrontend" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7ffeef215640 (LWP 14567)]
0x00007fffd002c5ea in P8PLATFORM::CThread::ThreadHandler(void*) () from /lib/x86_64-linux-gnu/libcec.so.6
(gdb) where
#0  0x00007fffd002c5ea in P8PLATFORM::CThread::ThreadHandler(void*) () at /lib/x86_64-linux-gnu/libcec.so.6
#1  0x00007ffff1eadb43 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
#2  0x00007ffff1f3fa00 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81
(gdb) thread apply all bt full

Thread 30 (Thread 0x7ffeef215640 (LWP 14567) "mythfrontend"):
#0  0x00007fffd002c5ea in P8PLATFORM::CThread::ThreadHandler(void*) () at /lib/x86_64-linux-gnu/libcec.so.6
#1  0x00007ffff1eadb43 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
        ret = <optimized out>
        pd = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140737488343600, 8290270341860128518, 140732910360128, 0, 140737252087888, 140737488343952, -8290798497951156474, -8290257035203348730}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
#2  0x00007ffff1f3fa00 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

...

Thread 1 (Thread 0x7fffea071bc0 (LWP 14534) "mythfrontend"):
#0  __GI__dl_debug_state () at ./elf/dl-debug.c:116
#1  0x00007ffff7fc59ba in _dl_close_worker (force=force@entry=false, map=<optimized out>, map=<optimized out>) at ./elf/dl-close.c:464
        nsid = <optimized out>
        any_tls = false
        nloaded = <optimized out>
        maps = 0x7fffffffcba0
        idx = <optimized out>
        done_index = <optimized out>
        unload_any = true
        scope_mem_left = false
        unload_global = 0
        first_loaded = <optimized out>
        r = 0x7ffff7ffe118 <_r_debug>
        tls_free_start = <optimized out>
        tls_free_end = <optimized out>
        dl_close_state = pending
        __PRETTY_FUNCTION__ = "_dl_close_worker"
#2  0x00007ffff7fc62a2 in _dl_close_worker (force=false, map=0x555556a4b2a0) at ./elf/dl-close.c:150
        dl_close_state = pending
        __PRETTY_FUNCTION__ = "_dl_close_worker"
        map = 0x555556a4b2a0
#3  _dl_close (_map=0x555556a4b2a0) at ./elf/dl-close.c:818
        map = 0x555556a4b2a0
#4  0x00007ffff1f8dc28 in __GI__dl_catch_exception (exception=exception@entry=0x7fffffffd540, operate=<optimized out>, args=<optimized out>) at ./elf/dl-error-skeleton.c:208
        errcode = 21845
        c = {exception = 0x7fffffffd540, errcode = 0x7fffffffd44c, env = {{__jmpbuf = {140737488344471, -8290270341116758266, -264, 93825013611808, 140737336177626, 140737339440120, -8290270341099981050, -8290257189644437754}, __mask_was_saved = 0, __saved_mask = {__val = {0 <repeats 16 times>}}}}}
        old = 0x0
#5  0x00007ffff1f8dcf3 in __GI__dl_catch_error (objname=0x7fffffffd598, errstring=0x7fffffffd5a0, mallocedp=0x7fffffffd597, operate=<optimized out>, args=<optimized out>) at ./elf/dl-error-skeleton.c:227
        exception = {objname = 0x0, errstring = 0x0, message_buffer = 0x0}
        errorcode = <optimized out>
#6  0x00007ffff1ea91ae in _dlerror_run (operate=<optimized out>, args=<optimized out>) at ./dlfcn/dlerror.c:138
        result = <optimized out>
        objname = 0x0
        errstring = 0x0
        malloced = false
        errcode = <optimized out>
#7  0x00007ffff1ea8ed8 in __dlclose (handle=<optimized out>) at ./dlfcn/dlclose.c:31
#8  0x00007ffff6eafcec in UnloadLibCec(CEC::ICECAdapter*) (device=0x5555569b7d20) at /usr/include/libcec/cecloader.h:155
        DestroyLibCec = <optimized out>
        __FUNCTION__ = "Close"
#9  MythCECAdapter::Close() (this=0x555555d40438) at devices/mythcecadapter.cpp:263
        __FUNCTION__ = "Close"
#10 0x00007ffff6eb1058 in MythCECAdapter::Open(MythMainWindow*) (this=0x555555d40438, Window=<optimized out>) at devices/mythcecadapter.cpp:197
        __FUNCTION__ = "Open"
        defaultDevice = {d = 0x555556c30f10}
        base_dev = {d = 0x555556b1bfb0}
        hdmi_port = {d = 0x555556b21900}
        configuration = {clientVersion = 393218, strDeviceName = "MythTV\000\000\240\006\240\006\000\000", deviceTypes = {types = {CEC::CEC_DEVICE_TYPE_PLAYBACK_DEVICE, CEC::CEC_DEVICE_TYPE_RESERVED, CEC::CEC_DEVICE_TYPE_RESERVED, CEC::CEC_DEVICE_TYPE_RESERVED, CEC::CEC_DEVICE_TYPE_RESERVED}}, bAutodetectAddress = 0 '\000', iPhysicalAddress = 0, baseDevice = CEC::CECDEVICE_TV, iHDMIPort = 1 '\001', tvVendor = 0, wakeDevices = {primary = CEC::CECDEVICE_TV, addresses = {1, 0 <repeats 15 times>}}, powerOffDevices = {primary = CEC::CECDEVICE_UNREGISTERED, addresses = {0 <repeats 15 times>, 1}}, serverVersion = 393218, bGetSettingsFromROM = 0 '\000', bActivateSource = 1 '\001', bPowerOffOnStandby = 1 '\001', callbackParam = 0x555555d40438, callbacks = 0x555555d40440, logicalAddresses = {primary = CEC::CECDEVICE_UNREGISTERED, addresses = {0 <repeats 16 times>}}, iFirmwareVersion = 65535, strDeviceLanguage = "eng", iFirmwareBuildDate = 0, bMonitorOnly = 0 '\000', cecVersion = CEC::CEC_VERSION_1_4, adapterType = CEC::ADAPTERTYPE_UNKNOWN, comboKey = CEC::CEC_USER_CONTROL_CODE_STOP, iComboKeyTimeoutMs = 1000, iButtonRepeatRateMs = 0, iButtonReleaseDelayMs = 500, iDoubleTapTimeoutMs = 200, bAutoWakeAVR = 0 '\000', bAutoPowerOn = 2 '\002'}
        display = <optimized out>
        devices = <optimized out>
        num_devices = <optimized out>
        devicenum = <optimized out>
        find = <optimized out>
        comm = {d = 0x7ffff271b5f0 <QListData::shared_null>}
        path = {d = 0x7ffff271aae0 <QArrayData::shared_null>}
        actions = {i = <optimized out>}
#11 0x00007ffff2667783 in  () at /lib/x86_64-linux-gnu/libQt5Core.so.5
#12 0x00007ffff6d57d70 in MythMainWindow::Init(bool) (this=this@entry=0x555555cad0e0, MayReInit=MayReInit@entry=false) at mythmainwindow.cpp:766
        flags = {i = <optimized out>}
        inwindow = <optimized out>
        fullscreen = <optimized out>
        __FUNCTION__ = "Init"
        warningmsg = {d = 0x7ffff271aae0 <QArrayData::shared_null>}
#13 0x00005555555e1b1e in main(int, char**) (argc=<optimized out>, argv=<optimized out>) at main.cpp:2044
        bPromptForBackend = <optimized out>
        bBypassAutoDiscovery = <optimized out>
        cmdline = {<MythCommandLineParser> = {_vptr.MythCommandLineParser = 0x5555558ef4d0 <vtable for MythFrontendCommandLineParser+16>, m_appname = {d = 0x555555a88870}, m_optionedArgs = {d = 0x555555a88e10}, m_namedArgs = {d = 0x555555a88d80}, m_passthroughActive = false, m_overridesImported = true, m_verbose = false}, <No data fields>}
        a = <incomplete type>
        callCleanup = {m_cleanFunction = 0x5555555ff9d0 <(anonymous namespace)::cleanup()>}
        signallist = {<QListSpecialMethods<int>> = {<No data fields>}, {p = {static shared_null = {ref = {atomic = {_q_value = {<std::__atomic_base<int>> = {static _S_alignment = 4, _M_i = -1}, static is_always_lock_free = true}}}, alloc = 0, begin = 0, end = 0, array = {0x0}}, d = 0x555555c21a60}, d = 0x555555c21a60}}
        retval = <optimized out>
        ResetSettings = false
        __FUNCTION__ = "main"
        fileprefix = {d = 0x555555ceb640}
        dir = {d_ptr = {d = 0x555555e7ce30}}
        bonjour = {d = 0x555555d95340}
        themename = {d = 0x555555ca9b20}
        themedir = {d = 0x555555cad050}
        mainWindow = 0x555555cad0e0
        mon = <optimized out>
        networkControl = <optimized out>
        themeUpdateChecker = std::unique_ptr<ThemeUpdateChecker> = {get() = 0x43}
        sysEventHandler = {<QObject> = {<No data fields>}, static staticMetaObject = {d = {superdata = {direct = 0x7ffff28c5160 <QObject::staticMetaObject>}, stringdata = 0x7ffff7daedc0 <qt_meta_stringdata_MythSystemEventHandler>, data = 0x7ffff7daed80 <qt_meta_data_MythSystemEventHandler>, static_metacall = 0x7ffff7cdebe0 <MythSystemEventHandler::qt_static_metacall(QObject*, QMetaObject::Call, int, void**)>, relatedMetaObjects = 0x0, extradata = 0x0}}}
        bcm = {<QObject> = {<No data fields>}, static staticMetaObject = {d = {superdata = {direct = 0x7ffff28c5160 <QObject::staticMetaObject>}, stringdata = 0x55555586ea00 <qt_meta_stringdata_BackendConnectionManager>, data = 0x55555586e9a0 <qt_meta_data_BackendConnectionManager>, static_metacall = 0x555555838ec0 <BackendConnectionManager::qt_static_metacall(QObject*, QMetaObject::Call, int, void**)>, relatedMetaObjects = 0x0, extradata = 0x0}}, m_reconnecting = 0x7ffff2f2f580, m_reconnectTimer = 0x7ffff2f35140, m_reconnectAgain = 160}
        housekeeping = <optimized out>
        ret = <optimized out>

Returning to master, I added --disable-libcec to ./configure and I can no longer trigger a segmentation fault.

@garybuhrmaster
Copy link
Contributor

Conjecture: Since this is in the libcec close/unload functionality, #299 (or the fix applied to MythTV) may be related.

@ulmus-scott
Copy link
Contributor Author

I had seen #299 and #299 (comment) seems to work:

diff --git a/mythtv/libs/libmythui/devices/mythcecadapter.cpp b/mythtv/libs/libmythui/devices/mythcecadapter.cpp
index b0c7f2e6a7..53a5bf17bc 100644
--- a/mythtv/libs/libmythui/devices/mythcecadapter.cpp
+++ b/mythtv/libs/libmythui/devices/mythcecadapter.cpp
@@ -261,10 +261,10 @@ void MythCECAdapter::Close(void)
         if (m_powerOffTVOnExit)
             HandleActions(PowerOffTV);
         m_adapter->Close();
-        UnloadLibCec(m_adapter);
+        //UnloadLibCec(m_adapter);
         // Workaround for bug in libcec/cecloader.h
         // MythTV issue #299, libcec issue #555
-        g_libCEC = nullptr;
+        //g_libCEC = nullptr;
         LOG(VB_GENERAL, LOG_INFO, LOC + "Closing down CEC.");
     }
     m_valid = false;

but that leaks memory.

Without dlclose():

Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x0000560095d3e4c0 in ?? ()
[Current thread is 1 (Thread 0x7fed15b1e640 (LWP 69943))]
(gdb) where
#0  0x0000560095d3e4c0 in  ()
#1  0x00007fed2c5a05ed in P8PLATFORM::CThread::ThreadHandler(void*) () at /lib/x86_64-linux-gnu/libcec.so.6
#2  0x00007fedf7df1b43 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
#3  0x00007fedf7e83a00 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Without CECDestroy (memory leak):

Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x00007f1100556263 in ?? ()
[Current thread is 1 (Thread 0x7f10cdb1e640 (LWP 71267))]
(gdb) where
#0  0x00007f1100556263 in  ()
#1  0x00000000000001f4 in  ()
#2  0x000055c7305af490 in  ()
#3  0x0000000000000000 in  ()

Without both it doesn't crash. This feels to me that it is memory corruption and libCEC is a red herring. I'll have to try running valgrind with a --compile-type=debug compile.

@benjsc
Copy link
Contributor

benjsc commented Jun 19, 2023 •
edited

I'm noticing the crash as well, this is on a FreeBSD 13.2 box.

Core was generated by `mythfrontend'.
Program terminated with signal SIGSEGV, Segmentation fault.
Sent by thr_kill() from pid 42210 and user 1002.
#0  0x00000008d4184fb0 in ?? ()
[Current thread is 1 (LWP 127140)]
(gdb) bt
#0  0x00000008d4184fb0 in ?? ()
#1  0x000000082f72da7a in ?? ()
#2  0x0000000000000000 in ?? ()

The stack trace clearly looks corrupted

@kmdewaal
Copy link
Contributor

Reproduced today (but only once....) with mythtv-setup.
Unfortunately the backtrace is completely meaningless:

(gdb) bt
#0  0x00007f6ec40beb8e in ??? ()
#1  0x00007f6e83058cdc in ??? ()
#2  0x00007f6e830586c0 in ??? ()
#3  0xffffffffffffff08 in ??? ()
#4  0x0000000000000000 in ??? ()

And this is from a debug build...
Could be related to UPnP.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@garybuhrmaster @benjsc @kmdewaal @ulmus-scott