"Could not roll back JDBC transaction"

Hi,

Trying to list all the datasets using the Java API leads to the following error when the group contains too many objects (apparently):

Could not load datasetsomero.InternalException
    serverStackTrace = "ome.conditions.InternalException:  Wrapped Exception: (org.springframework.transaction.TransactionSystemException):
                        Could not roll back JDBC transaction; nested exception is org.postgresql.util.PSQLException: This connection has been closed.
                        	at org.springframework.jdbc.datasource.DataSourceTransactionManager.doRollback(DataSourceTransactionManager.java:331)
                        	at org.springframework.transaction.support.AbstractPlatformTransactionManager.processRollback(AbstractPlatformTransactionManager.java:853)
                        	at org.springframework.transaction.support.AbstractPlatformTransactionManager.rollback(AbstractPlatformTransactionManager.java:830)
                        	at org.springframework.transaction.interceptor.TransactionAspectSupport.completeTransactionAfterThrowing(TransactionAspectSupport.java:536)
                        	at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:286)
                        	at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96)
                        	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
                        	at ome.tools.hibernate.ProxyCleanupFilter$Interceptor.invoke(ProxyCleanupFilter.java:249)
                        	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
                        	at ome.services.util.ServiceHandler.invoke(ServiceHandler.java:121)
                        	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
                        	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:213)
                        	at com.sun.proxy.$Proxy105.loadContainerHierarchy(Unknown Source)
                        	at jdk.internal.reflect.GeneratedMethodAccessor1171.invoke(Unknown Source)
                        	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
                        	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
                        	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:333)
                        	at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190)
                        	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
                        	at ome.security.basic.BasicSecurityWiring.invoke(BasicSecurityWiring.java:93)
                        	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
                        	at ome.services.blitz.fire.AopContextInitializer.invoke(AopContextInitializer.java:43)
                        	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
                        	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:213)
                        	at com.sun.proxy.$Proxy105.loadContainerHierarchy(Unknown Source)
                        	at jdk.internal.reflect.GeneratedMethodAccessor1249.invoke(Unknown Source)
                        	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
                        	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
                        	at ome.services.blitz.util.IceMethodInvoker.invoke(IceMethodInvoker.java:172)
                        	at ome.services.throttling.Callback.run(Callback.java:56)
                        	at ome.services.throttling.InThreadThrottlingStrategy.callInvokerOnRawArgs(InThreadThrottlingStrategy.java:56)
                        	at ome.services.blitz.impl.AbstractAmdServant.callInvokerOnRawArgs(AbstractAmdServant.java:140)
                        	at ome.services.blitz.impl.ContainerI.loadContainerHierarchy_async(ContainerI.java:177)
                        	at jdk.internal.reflect.GeneratedMethodAccessor1248.invoke(Unknown Source)
                        	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
                        	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
                        	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:333)
                        	at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190)
                        	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
                        	at omero.cmd.CallContext.invoke(CallContext.java:85)
                        	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
                        	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:213)
                        	at com.sun.proxy.$Proxy106.loadContainerHierarchy_async(Unknown Source)
                        	at omero.api._IContainerTie.loadContainerHierarchy_async(_IContainerTie.java:120)
                        	at omero.api._IContainerDisp.___loadContainerHierarchy(_IContainerDisp.java:537)
                        	at omero.api._IContainerDisp.__dispatch(_IContainerDisp.java:980)
                        	at IceInternal.Incoming.invoke(Incoming.java:221)
                        	at Ice.ConnectionI.invokeAll(ConnectionI.java:2536)
                        	at Ice.ConnectionI.dispatch(ConnectionI.java:1145)
                        	at Ice.ConnectionI.message(ConnectionI.java:1056)
                        	at IceInternal.ThreadPool.run(ThreadPool.java:395)
                        	at IceInternal.ThreadPool.access$300(ThreadPool.java:12)
                        	at IceInternal.ThreadPool$EventHandlerThread.run(ThreadPool.java:832)
                        	at java.base/java.lang.Thread.run(Thread.java:834)
                        "

This is what I get with:

gateway.getBrowseFacility().getDatasets(ctx);

If I switch to a group with fewer datasets, or specify a user, I can retrieve a list.
Is there a way to fix (or bypass) this if we want the lists of all the datasets in the current group?

Hi Pierre.
I’d expect some kind of timeout exception if that’s caused by too many datasets. How long does it take until you get the error, or do you get it straight away?
An alternative would be to use an HQL query, e. g.

        IQueryPrx qs = gateway.getQueryService(ctx);
        ParametersI p = new ParametersI();
        List<IObject> results = qs.findAllByQuery("select d from Dataset d", p);
        for (IObject r : results) {
            DatasetData d = new DatasetData((Dataset) r);
            System.out.println(d.getName());
        }

Wonder if you’ll get the same error using HQL…

Note: The datasets you’ll get with this query won’t be fully initialized, e. g. calling getImages() on them will fail.

Regards,
Dominik

The first time, it happens after about 6 minutes, but after that, it fails immediately.

I just tried it, and it’s fast! It takes about 1-2 seconds, so no timeout there!
Thank you!

I think it’s also the case with the BrowseFacility::getDatasets() method. I just tried it in a group with fewer datasets, and if I use getImages() on a dataset retrieved that way, I get an empty list, but if I use BrowseFacility::getImagesForDatasets(), I get the proper list.
However, if I retrieve a DatasetData directly through its ID, it is properly initialized, and thus getImages() works, which I didn’t know (we use getImagesForDatasets() by default to avoid problems).

I would also have another question about a problem retrieving Tables when there are (unloaded) ROIs in a column, but as it is unrelated, I should create a new topic after I run some more tests…

Once again, thank you!

That’s interesting. A lot of methods in the gateway are based on HQL queries under the hood, so I wouldn’t actually have expected much of a difference there. Clearly have to review the gateway methods. I hope for now you can use the HQL workaround. I also noticed that the docs aren’t very clear about the returned objects, to which level they are initialized. I just openend two respective issues on github to keep track of that: Review/Improve method performance and Improve documentation wrt initialization state of returned objects

Kind Regards,
Dominik

2 Likes