2017-03-09 09:57:40 INFO SparkContext:54 - Running Spark version 2.0.2 2017-03-09 09:57:41 WARN NativeCodeLoader:62 - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2017-03-09 09:57:41 WARN SparkConf:66 - SPARK_CLASSPATH was detected (set to '/Users/cseed/hail/build/libs/hail-all-spark.jar'). This is deprecated in Spark 1.0+. Please instead use: - ./spark-submit with --driver-class-path to augment the driver classpath - spark.executor.extraClassPath to augment the executor classpath 2017-03-09 09:57:41 WARN SparkConf:66 - Setting 'spark.executor.extraClassPath' to '/Users/cseed/hail/build/libs/hail-all-spark.jar' as a work-around. 2017-03-09 09:57:41 WARN SparkConf:66 - Setting 'spark.driver.extraClassPath' to '/Users/cseed/hail/build/libs/hail-all-spark.jar' as a work-around. 2017-03-09 09:57:41 INFO SecurityManager:54 - Changing view acls to: cseed 2017-03-09 09:57:41 INFO SecurityManager:54 - Changing modify acls to: cseed 2017-03-09 09:57:41 INFO SecurityManager:54 - Changing view acls groups to: 2017-03-09 09:57:41 INFO SecurityManager:54 - Changing modify acls groups to: 2017-03-09 09:57:41 INFO SecurityManager:54 - SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(cseed); groups with view permissions: Set(); users with modify permissions: Set(cseed); groups with modify permissions: Set() 2017-03-09 09:57:41 INFO Utils:54 - Successfully started service 'sparkDriver' on port 50780. 2017-03-09 09:57:41 INFO SparkEnv:54 - Registering MapOutputTracker 2017-03-09 09:57:41 INFO SparkEnv:54 - Registering BlockManagerMaster 2017-03-09 09:57:41 INFO DiskBlockManager:54 - Created local directory at /private/var/folders/f7/v68hyflx78q5nyy2w434w2jxrzcs23/T/blockmgr-a01f0ef6-b81a-4300-b859-0dbaf95e407f 2017-03-09 09:57:41 INFO MemoryStore:54 - MemoryStore started with capacity 366.3 MB 2017-03-09 09:57:41 INFO SparkEnv:54 - Registering OutputCommitCoordinator 2017-03-09 09:57:41 INFO log:186 - Logging initialized @1966ms 2017-03-09 09:57:41 INFO Server:327 - jetty-9.2.z-SNAPSHOT 2017-03-09 09:57:41 INFO ContextHandler:744 - Started o.s.j.s.ServletContextHandler@632ab972{/jobs,null,AVAILABLE} 2017-03-09 09:57:41 INFO ContextHandler:744 - Started o.s.j.s.ServletContextHandler@24947bc8{/jobs/json,null,AVAILABLE} 2017-03-09 09:57:41 INFO ContextHandler:744 - Started o.s.j.s.ServletContextHandler@16528c0a{/jobs/job,null,AVAILABLE} 2017-03-09 09:57:41 INFO ContextHandler:744 - Started o.s.j.s.ServletContextHandler@3eccd1be{/jobs/job/json,null,AVAILABLE} 2017-03-09 09:57:41 INFO ContextHandler:744 - Started o.s.j.s.ServletContextHandler@89b0e3c{/stages,null,AVAILABLE} 2017-03-09 09:57:41 INFO ContextHandler:744 - Started o.s.j.s.ServletContextHandler@f567edc{/stages/json,null,AVAILABLE} 2017-03-09 09:57:41 INFO ContextHandler:744 - Started o.s.j.s.ServletContextHandler@d6681e0{/stages/stage,null,AVAILABLE} 2017-03-09 09:57:41 INFO ContextHandler:744 - Started o.s.j.s.ServletContextHandler@55fff0d3{/stages/stage/json,null,AVAILABLE} 2017-03-09 09:57:41 INFO ContextHandler:744 - Started o.s.j.s.ServletContextHandler@5c5febe{/stages/pool,null,AVAILABLE} 2017-03-09 09:57:41 INFO ContextHandler:744 - Started o.s.j.s.ServletContextHandler@3959d044{/stages/pool/json,null,AVAILABLE} 2017-03-09 09:57:41 INFO ContextHandler:744 - Started o.s.j.s.ServletContextHandler@892f0e{/storage,null,AVAILABLE} 2017-03-09 09:57:41 INFO ContextHandler:744 - Started o.s.j.s.ServletContextHandler@2a5a3ea4{/storage/json,null,AVAILABLE} 2017-03-09 09:57:41 INFO ContextHandler:744 - Started o.s.j.s.ServletContextHandler@c0155f{/storage/rdd,null,AVAILABLE} 2017-03-09 09:57:41 INFO ContextHandler:744 - Started o.s.j.s.ServletContextHandler@776c3ef3{/storage/rdd/json,null,AVAILABLE} 2017-03-09 09:57:41 INFO ContextHandler:744 - Started o.s.j.s.ServletContextHandler@3ed49e4f{/environment,null,AVAILABLE} 2017-03-09 09:57:41 INFO ContextHandler:744 - Started o.s.j.s.ServletContextHandler@4500282f{/environment/json,null,AVAILABLE} 2017-03-09 09:57:41 INFO ContextHandler:744 - Started o.s.j.s.ServletContextHandler@456aa73d{/executors,null,AVAILABLE} 2017-03-09 09:57:41 INFO ContextHandler:744 - Started o.s.j.s.ServletContextHandler@53e792c5{/executors/json,null,AVAILABLE} 2017-03-09 09:57:41 INFO ContextHandler:744 - Started o.s.j.s.ServletContextHandler@495b5810{/executors/threadDump,null,AVAILABLE} 2017-03-09 09:57:41 INFO ContextHandler:744 - Started o.s.j.s.ServletContextHandler@d5e5044{/executors/threadDump/json,null,AVAILABLE} 2017-03-09 09:57:41 INFO ContextHandler:744 - Started o.s.j.s.ServletContextHandler@1d1d5d9d{/static,null,AVAILABLE} 2017-03-09 09:57:41 INFO ContextHandler:744 - Started o.s.j.s.ServletContextHandler@72038541{/,null,AVAILABLE} 2017-03-09 09:57:41 INFO ContextHandler:744 - Started o.s.j.s.ServletContextHandler@618bd8c9{/api,null,AVAILABLE} 2017-03-09 09:57:41 INFO ContextHandler:744 - Started o.s.j.s.ServletContextHandler@1e2868cc{/stages/stage/kill,null,AVAILABLE} 2017-03-09 09:57:41 WARN AbstractLifeCycle:212 - FAILED ServerConnector@7a830ff4{HTTP/1.1}{0.0.0.0:4040}: java.net.BindException: Address already in use java.net.BindException: Address already in use at sun.nio.ch.Net.bind0(Native Method) at sun.nio.ch.Net.bind(Net.java:437) at sun.nio.ch.Net.bind(Net.java:429) at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223) at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74) at org.spark_project.jetty.server.ServerConnector.open(ServerConnector.java:321) at org.spark_project.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:80) at org.spark_project.jetty.server.ServerConnector.doStart(ServerConnector.java:236) at org.spark_project.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) at org.spark_project.jetty.server.Server.doStart(Server.java:366) at org.spark_project.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) at org.apache.spark.ui.JettyUtils$.org$apache$spark$ui$JettyUtils$$connect$1(JettyUtils.scala:306) at org.apache.spark.ui.JettyUtils$$anonfun$5.apply(JettyUtils.scala:316) at org.apache.spark.ui.JettyUtils$$anonfun$5.apply(JettyUtils.scala:316) at org.apache.spark.util.Utils$$anonfun$startServiceOnPort$1.apply$mcVI$sp(Utils.scala:2171) at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:160) at org.apache.spark.util.Utils$.startServiceOnPort(Utils.scala:2162) at org.apache.spark.ui.JettyUtils$.startJettyServer(JettyUtils.scala:316) at org.apache.spark.ui.WebUI.bind(WebUI.scala:139) at org.apache.spark.SparkContext$$anonfun$10.apply(SparkContext.scala:448) at org.apache.spark.SparkContext$$anonfun$10.apply(SparkContext.scala:448) at scala.Option.foreach(Option.scala:257) at org.apache.spark.SparkContext.(SparkContext.scala:448) at is.hail.HailContext$.configureAndCreateSparkContext(HailContext.scala:84) at is.hail.HailContext$.apply(HailContext.scala:159) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:237) at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357) at py4j.Gateway.invoke(Gateway.java:280) at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132) at py4j.commands.CallCommand.execute(CallCommand.java:79) at py4j.GatewayConnection.run(GatewayConnection.java:214) at java.lang.Thread.run(Thread.java:745) 2017-03-09 09:57:41 WARN AbstractLifeCycle:212 - FAILED org.spark_project.jetty.server.Server@79277fa: java.net.BindException: Address already in use java.net.BindException: Address already in use at sun.nio.ch.Net.bind0(Native Method) at sun.nio.ch.Net.bind(Net.java:437) at sun.nio.ch.Net.bind(Net.java:429) at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223) at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74) at org.spark_project.jetty.server.ServerConnector.open(ServerConnector.java:321) at org.spark_project.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:80) at org.spark_project.jetty.server.ServerConnector.doStart(ServerConnector.java:236) at org.spark_project.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) at org.spark_project.jetty.server.Server.doStart(Server.java:366) at org.spark_project.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) at org.apache.spark.ui.JettyUtils$.org$apache$spark$ui$JettyUtils$$connect$1(JettyUtils.scala:306) at org.apache.spark.ui.JettyUtils$$anonfun$5.apply(JettyUtils.scala:316) at org.apache.spark.ui.JettyUtils$$anonfun$5.apply(JettyUtils.scala:316) at org.apache.spark.util.Utils$$anonfun$startServiceOnPort$1.apply$mcVI$sp(Utils.scala:2171) at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:160) at org.apache.spark.util.Utils$.startServiceOnPort(Utils.scala:2162) at org.apache.spark.ui.JettyUtils$.startJettyServer(JettyUtils.scala:316) at org.apache.spark.ui.WebUI.bind(WebUI.scala:139) at org.apache.spark.SparkContext$$anonfun$10.apply(SparkContext.scala:448) at org.apache.spark.SparkContext$$anonfun$10.apply(SparkContext.scala:448) at scala.Option.foreach(Option.scala:257) at org.apache.spark.SparkContext.(SparkContext.scala:448) at is.hail.HailContext$.configureAndCreateSparkContext(HailContext.scala:84) at is.hail.HailContext$.apply(HailContext.scala:159) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:237) at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357) at py4j.Gateway.invoke(Gateway.java:280) at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132) at py4j.commands.CallCommand.execute(CallCommand.java:79) at py4j.GatewayConnection.run(GatewayConnection.java:214) at java.lang.Thread.run(Thread.java:745) 2017-03-09 09:57:41 INFO ServerConnector:306 - Stopped ServerConnector@7a830ff4{HTTP/1.1}{0.0.0.0:4040} 2017-03-09 09:57:41 INFO ContextHandler:865 - Stopped o.s.j.s.ServletContextHandler@1e2868cc{/stages/stage/kill,null,UNAVAILABLE} 2017-03-09 09:57:41 INFO ContextHandler:865 - Stopped o.s.j.s.ServletContextHandler@618bd8c9{/api,null,UNAVAILABLE} 2017-03-09 09:57:41 INFO ContextHandler:865 - Stopped o.s.j.s.ServletContextHandler@72038541{/,null,UNAVAILABLE} 2017-03-09 09:57:41 INFO ContextHandler:865 - Stopped o.s.j.s.ServletContextHandler@1d1d5d9d{/static,null,UNAVAILABLE} 2017-03-09 09:57:41 INFO ContextHandler:865 - Stopped o.s.j.s.ServletContextHandler@d5e5044{/executors/threadDump/json,null,UNAVAILABLE} 2017-03-09 09:57:41 INFO ContextHandler:865 - Stopped o.s.j.s.ServletContextHandler@495b5810{/executors/threadDump,null,UNAVAILABLE} 2017-03-09 09:57:41 INFO ContextHandler:865 - Stopped o.s.j.s.ServletContextHandler@53e792c5{/executors/json,null,UNAVAILABLE} 2017-03-09 09:57:41 INFO ContextHandler:865 - Stopped o.s.j.s.ServletContextHandler@456aa73d{/executors,null,UNAVAILABLE} 2017-03-09 09:57:41 INFO ContextHandler:865 - Stopped o.s.j.s.ServletContextHandler@4500282f{/environment/json,null,UNAVAILABLE} 2017-03-09 09:57:41 INFO ContextHandler:865 - Stopped o.s.j.s.ServletContextHandler@3ed49e4f{/environment,null,UNAVAILABLE} 2017-03-09 09:57:41 INFO ContextHandler:865 - Stopped o.s.j.s.ServletContextHandler@776c3ef3{/storage/rdd/json,null,UNAVAILABLE} 2017-03-09 09:57:41 INFO ContextHandler:865 - Stopped o.s.j.s.ServletContextHandler@c0155f{/storage/rdd,null,UNAVAILABLE} 2017-03-09 09:57:41 INFO ContextHandler:865 - Stopped o.s.j.s.ServletContextHandler@2a5a3ea4{/storage/json,null,UNAVAILABLE} 2017-03-09 09:57:41 INFO ContextHandler:865 - Stopped o.s.j.s.ServletContextHandler@892f0e{/storage,null,UNAVAILABLE} 2017-03-09 09:57:41 INFO ContextHandler:865 - Stopped o.s.j.s.ServletContextHandler@3959d044{/stages/pool/json,null,UNAVAILABLE} 2017-03-09 09:57:41 INFO ContextHandler:865 - Stopped o.s.j.s.ServletContextHandler@5c5febe{/stages/pool,null,UNAVAILABLE} 2017-03-09 09:57:41 INFO ContextHandler:865 - Stopped o.s.j.s.ServletContextHandler@55fff0d3{/stages/stage/json,null,UNAVAILABLE} 2017-03-09 09:57:41 INFO ContextHandler:865 - Stopped o.s.j.s.ServletContextHandler@d6681e0{/stages/stage,null,UNAVAILABLE} 2017-03-09 09:57:41 INFO ContextHandler:865 - Stopped o.s.j.s.ServletContextHandler@f567edc{/stages/json,null,UNAVAILABLE} 2017-03-09 09:57:41 INFO ContextHandler:865 - Stopped o.s.j.s.ServletContextHandler@89b0e3c{/stages,null,UNAVAILABLE} 2017-03-09 09:57:41 INFO ContextHandler:865 - Stopped o.s.j.s.ServletContextHandler@3eccd1be{/jobs/job/json,null,UNAVAILABLE} 2017-03-09 09:57:41 INFO ContextHandler:865 - Stopped o.s.j.s.ServletContextHandler@16528c0a{/jobs/job,null,UNAVAILABLE} 2017-03-09 09:57:41 INFO ContextHandler:865 - Stopped o.s.j.s.ServletContextHandler@24947bc8{/jobs/json,null,UNAVAILABLE} 2017-03-09 09:57:41 INFO ContextHandler:865 - Stopped o.s.j.s.ServletContextHandler@632ab972{/jobs,null,UNAVAILABLE} 2017-03-09 09:57:41 WARN Utils:66 - Service 'SparkUI' could not bind on port 4040. Attempting port 4041. 2017-03-09 09:57:41 INFO Server:327 - jetty-9.2.z-SNAPSHOT 2017-03-09 09:57:41 INFO ContextHandler:744 - Started o.s.j.s.ServletContextHandler@632ab972{/jobs,null,AVAILABLE} 2017-03-09 09:57:41 INFO ContextHandler:744 - Started o.s.j.s.ServletContextHandler@24947bc8{/jobs/json,null,AVAILABLE} 2017-03-09 09:57:41 INFO ContextHandler:744 - Started o.s.j.s.ServletContextHandler@16528c0a{/jobs/job,null,AVAILABLE} 2017-03-09 09:57:41 INFO ContextHandler:744 - Started o.s.j.s.ServletContextHandler@3eccd1be{/jobs/job/json,null,AVAILABLE} 2017-03-09 09:57:41 INFO ContextHandler:744 - Started o.s.j.s.ServletContextHandler@89b0e3c{/stages,null,AVAILABLE} 2017-03-09 09:57:41 INFO ContextHandler:744 - Started o.s.j.s.ServletContextHandler@f567edc{/stages/json,null,AVAILABLE} 2017-03-09 09:57:41 INFO ContextHandler:744 - Started o.s.j.s.ServletContextHandler@d6681e0{/stages/stage,null,AVAILABLE} 2017-03-09 09:57:41 INFO ContextHandler:744 - Started o.s.j.s.ServletContextHandler@55fff0d3{/stages/stage/json,null,AVAILABLE} 2017-03-09 09:57:41 INFO ContextHandler:744 - Started o.s.j.s.ServletContextHandler@5c5febe{/stages/pool,null,AVAILABLE} 2017-03-09 09:57:41 INFO ContextHandler:744 - Started o.s.j.s.ServletContextHandler@3959d044{/stages/pool/json,null,AVAILABLE} 2017-03-09 09:57:41 INFO ContextHandler:744 - Started o.s.j.s.ServletContextHandler@892f0e{/storage,null,AVAILABLE} 2017-03-09 09:57:41 INFO ContextHandler:744 - Started o.s.j.s.ServletContextHandler@2a5a3ea4{/storage/json,null,AVAILABLE} 2017-03-09 09:57:41 INFO ContextHandler:744 - Started o.s.j.s.ServletContextHandler@c0155f{/storage/rdd,null,AVAILABLE} 2017-03-09 09:57:41 INFO ContextHandler:744 - Started o.s.j.s.ServletContextHandler@776c3ef3{/storage/rdd/json,null,AVAILABLE} 2017-03-09 09:57:41 INFO ContextHandler:744 - Started o.s.j.s.ServletContextHandler@3ed49e4f{/environment,null,AVAILABLE} 2017-03-09 09:57:41 INFO ContextHandler:744 - Started o.s.j.s.ServletContextHandler@4500282f{/environment/json,null,AVAILABLE} 2017-03-09 09:57:41 INFO ContextHandler:744 - Started o.s.j.s.ServletContextHandler@456aa73d{/executors,null,AVAILABLE} 2017-03-09 09:57:41 INFO ContextHandler:744 - Started o.s.j.s.ServletContextHandler@53e792c5{/executors/json,null,AVAILABLE} 2017-03-09 09:57:41 INFO ContextHandler:744 - Started o.s.j.s.ServletContextHandler@495b5810{/executors/threadDump,null,AVAILABLE} 2017-03-09 09:57:41 INFO ContextHandler:744 - Started o.s.j.s.ServletContextHandler@d5e5044{/executors/threadDump/json,null,AVAILABLE} 2017-03-09 09:57:41 INFO ContextHandler:744 - Started o.s.j.s.ServletContextHandler@1d1d5d9d{/static,null,AVAILABLE} 2017-03-09 09:57:41 INFO ContextHandler:744 - Started o.s.j.s.ServletContextHandler@72038541{/,null,AVAILABLE} 2017-03-09 09:57:41 INFO ContextHandler:744 - Started o.s.j.s.ServletContextHandler@618bd8c9{/api,null,AVAILABLE} 2017-03-09 09:57:41 INFO ContextHandler:744 - Started o.s.j.s.ServletContextHandler@1e2868cc{/stages/stage/kill,null,AVAILABLE} 2017-03-09 09:57:41 INFO ServerConnector:266 - Started ServerConnector@5e7cf171{HTTP/1.1}{0.0.0.0:4041} 2017-03-09 09:57:41 INFO Server:379 - Started @2071ms 2017-03-09 09:57:41 INFO Utils:54 - Successfully started service 'SparkUI' on port 4041. 2017-03-09 09:57:41 INFO SparkUI:54 - Bound SparkUI to 0.0.0.0, and started at http://192.168.200.115:4041 2017-03-09 09:57:41 INFO Executor:54 - Starting executor ID driver on host localhost 2017-03-09 09:57:41 INFO Utils:54 - Successfully started service 'org.apache.spark.network.netty.NettyBlockTransferService' on port 50781. 2017-03-09 09:57:41 INFO NettyBlockTransferService:54 - Server created on 192.168.200.115:50781 2017-03-09 09:57:41 INFO BlockManagerMaster:54 - Registering BlockManager BlockManagerId(driver, 192.168.200.115, 50781) 2017-03-09 09:57:41 INFO BlockManagerMasterEndpoint:54 - Registering block manager 192.168.200.115:50781 with 366.3 MB RAM, BlockManagerId(driver, 192.168.200.115, 50781) 2017-03-09 09:57:41 INFO BlockManagerMaster:54 - Registered BlockManager BlockManagerId(driver, 192.168.200.115, 50781) 2017-03-09 09:57:42 INFO ContextHandler:744 - Started o.s.j.s.ServletContextHandler@7da2320b{/metrics/json,null,AVAILABLE} 2017-03-09 09:57:42 INFO Hail:15 - SparkUI: http://192.168.200.115:4041 2017-03-09 09:57:42 INFO Hail:176 - Spark properties: spark.driver.port=50780, spark.ui.showConsoleProgress=false, spark.driver.host=192.168.200.115, spark.executor.id=driver, spark.app.id=local-1489078661873, spark.sql.files.openCostInBytes=1099511627776, spark.serializer=org.apache.spark.serializer.KryoSerializer, spark.hadoop.parquet.block.size=1099511627776, spark.hadoop.mapreduce.input.fileinputformat.split.minsize=1048576, spark.hadoop.io.compression.codecs=org.apache.hadoop.io.compress.DefaultCodec,is.hail.io.compress.BGzipCodec,org.apache.hadoop.io.compress.GzipCodec, spark.sql.parquet.compression.codec=snappy, spark.driver.extraClassPath=/Users/cseed/hail/build/libs/hail-all-spark.jar, spark.executor.extraClassPath=/Users/cseed/hail/build/libs/hail-all-spark.jar, spark.master=local[*], spark.submit.deployMode=client, spark.app.name=Hail, spark.sql.files.maxPartitionBytes=1099511627776 2017-03-09 09:58:28 ERROR Hail:65 - hail: fatal: No function found with name `+' and argument types (Array[Array[Int]], Array[Array[Double]]) :1:[[1, 2]] + [[1.0, 2.0]] ^ From is.hail.utils.package$FatalException: No function found with name `+' and argument types (Array[Array[Int]], Array[Array[Double]]) :1:[[1, 2]] + [[1.0, 2.0]] ^ at is.hail.utils.ErrorHandling$class.fatal(ErrorHandling.scala:5) at is.hail.utils.package$.fatal(package.scala:20) at is.hail.expr.ParserUtils$.error(Parser.scala:24) at is.hail.expr.AST.parseError(AST.scala:229) at is.hail.expr.Apply$$anonfun$typecheckThis$12.apply(AST.scala:430) at is.hail.expr.Apply$$anonfun$typecheckThis$12.apply(AST.scala:430) at is.hail.utils.EitherOps$.valueOr$extension(EitherIsAMonad.scala:21) at is.hail.expr.Apply.typecheckThis(AST.scala:430) at is.hail.expr.AST.typecheckThis(AST.scala:220) at is.hail.expr.AST.typecheck(AST.scala:226) at is.hail.expr.Apply.typecheck(AST.scala:509) at is.hail.expr.Parser$.is$hail$expr$Parser$$eval(Parser.scala:57) at is.hail.expr.Parser$.parseExpr(Parser.scala:67) at is.hail.HailContext.eval(HailContext.scala:581) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:237) at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357) at py4j.Gateway.invoke(Gateway.java:280) at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132) at py4j.commands.CallCommand.execute(CallCommand.java:79) at py4j.GatewayConnection.run(GatewayConnection.java:214) at java.lang.Thread.run(Thread.java:745) 2017-03-09 10:09:19 INFO SparkContext:54 - Invoking stop() from shutdown hook 2017-03-09 10:09:19 INFO ServerConnector:306 - Stopped ServerConnector@5e7cf171{HTTP/1.1}{0.0.0.0:4041} 2017-03-09 10:09:19 INFO ContextHandler:865 - Stopped o.s.j.s.ServletContextHandler@1e2868cc{/stages/stage/kill,null,UNAVAILABLE} 2017-03-09 10:09:19 INFO ContextHandler:865 - Stopped o.s.j.s.ServletContextHandler@618bd8c9{/api,null,UNAVAILABLE} 2017-03-09 10:09:19 INFO ContextHandler:865 - Stopped o.s.j.s.ServletContextHandler@72038541{/,null,UNAVAILABLE} 2017-03-09 10:09:19 INFO ContextHandler:865 - Stopped o.s.j.s.ServletContextHandler@1d1d5d9d{/static,null,UNAVAILABLE} 2017-03-09 10:09:19 INFO ContextHandler:865 - Stopped o.s.j.s.ServletContextHandler@d5e5044{/executors/threadDump/json,null,UNAVAILABLE} 2017-03-09 10:09:19 INFO ContextHandler:865 - Stopped o.s.j.s.ServletContextHandler@495b5810{/executors/threadDump,null,UNAVAILABLE} 2017-03-09 10:09:19 INFO ContextHandler:865 - Stopped o.s.j.s.ServletContextHandler@53e792c5{/executors/json,null,UNAVAILABLE} 2017-03-09 10:09:19 INFO ContextHandler:865 - Stopped o.s.j.s.ServletContextHandler@456aa73d{/executors,null,UNAVAILABLE} 2017-03-09 10:09:19 INFO ContextHandler:865 - Stopped o.s.j.s.ServletContextHandler@4500282f{/environment/json,null,UNAVAILABLE} 2017-03-09 10:09:19 INFO ContextHandler:865 - Stopped o.s.j.s.ServletContextHandler@3ed49e4f{/environment,null,UNAVAILABLE} 2017-03-09 10:09:19 INFO ContextHandler:865 - Stopped o.s.j.s.ServletContextHandler@776c3ef3{/storage/rdd/json,null,UNAVAILABLE} 2017-03-09 10:09:19 INFO ContextHandler:865 - Stopped o.s.j.s.ServletContextHandler@c0155f{/storage/rdd,null,UNAVAILABLE} 2017-03-09 10:09:19 INFO ContextHandler:865 - Stopped o.s.j.s.ServletContextHandler@2a5a3ea4{/storage/json,null,UNAVAILABLE} 2017-03-09 10:09:19 INFO ContextHandler:865 - Stopped o.s.j.s.ServletContextHandler@892f0e{/storage,null,UNAVAILABLE} 2017-03-09 10:09:19 INFO ContextHandler:865 - Stopped o.s.j.s.ServletContextHandler@3959d044{/stages/pool/json,null,UNAVAILABLE} 2017-03-09 10:09:19 INFO ContextHandler:865 - Stopped o.s.j.s.ServletContextHandler@5c5febe{/stages/pool,null,UNAVAILABLE} 2017-03-09 10:09:19 INFO ContextHandler:865 - Stopped o.s.j.s.ServletContextHandler@55fff0d3{/stages/stage/json,null,UNAVAILABLE} 2017-03-09 10:09:19 INFO ContextHandler:865 - Stopped o.s.j.s.ServletContextHandler@d6681e0{/stages/stage,null,UNAVAILABLE} 2017-03-09 10:09:19 INFO ContextHandler:865 - Stopped o.s.j.s.ServletContextHandler@f567edc{/stages/json,null,UNAVAILABLE} 2017-03-09 10:09:19 INFO ContextHandler:865 - Stopped o.s.j.s.ServletContextHandler@89b0e3c{/stages,null,UNAVAILABLE} 2017-03-09 10:09:19 INFO ContextHandler:865 - Stopped o.s.j.s.ServletContextHandler@3eccd1be{/jobs/job/json,null,UNAVAILABLE} 2017-03-09 10:09:19 INFO ContextHandler:865 - Stopped o.s.j.s.ServletContextHandler@16528c0a{/jobs/job,null,UNAVAILABLE} 2017-03-09 10:09:19 INFO ContextHandler:865 - Stopped o.s.j.s.ServletContextHandler@24947bc8{/jobs/json,null,UNAVAILABLE} 2017-03-09 10:09:19 INFO ContextHandler:865 - Stopped o.s.j.s.ServletContextHandler@632ab972{/jobs,null,UNAVAILABLE} 2017-03-09 10:09:19 INFO SparkUI:54 - Stopped Spark web UI at http://192.168.200.115:4041 2017-03-09 10:09:19 INFO MapOutputTrackerMasterEndpoint:54 - MapOutputTrackerMasterEndpoint stopped! 2017-03-09 10:09:19 INFO MemoryStore:54 - MemoryStore cleared 2017-03-09 10:09:19 INFO BlockManager:54 - BlockManager stopped 2017-03-09 10:09:19 INFO BlockManagerMaster:54 - BlockManagerMaster stopped 2017-03-09 10:09:19 INFO OutputCommitCoordinator$OutputCommitCoordinatorEndpoint:54 - OutputCommitCoordinator stopped! 2017-03-09 10:09:19 INFO SparkContext:54 - Successfully stopped SparkContext 2017-03-09 10:09:19 INFO ShutdownHookManager:54 - Shutdown hook called 2017-03-09 10:09:19 INFO ShutdownHookManager:54 - Deleting directory /private/var/folders/f7/v68hyflx78q5nyy2w434w2jxrzcs23/T/spark-d00a5b1c-e4e4-4eca-b73a-874679d1e2c8/pyspark-7cfb8090-a74b-4b20-b17d-3badb4444645 2017-03-09 10:09:19 INFO ShutdownHookManager:54 - Deleting directory /private/var/folders/f7/v68hyflx78q5nyy2w434w2jxrzcs23/T/spark-d00a5b1c-e4e4-4eca-b73a-874679d1e2c8