Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions ballista/scheduler/src/cache.rs
Original file line number Diff line number Diff line change
Expand Up @@ -64,9 +64,9 @@ pub(crate) fn contains_cache(plan: &LogicalPlan) -> bool {
.unwrap_or(false)
}

/// In the failure of a job that lost cache data mid-query. Rebuild-and-resume
/// is future work (see the design doc); the caches the job read are dropped,
/// so a retry rebuilds them.
/// In the failure of a job that lost cache data mid-query and could not rebuild
/// it (see the design doc); the caches the job read are dropped, so a retry
/// rebuilds them.
pub(crate) const CACHE_DATA_LOST: &str = "lost DataFrame::cache() data";

/// Replaces every `BallistaCacheNode` with its subplan, for when an operator
Expand Down
2 changes: 1 addition & 1 deletion ballista/scheduler/src/flight_sql.rs
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ impl<T: 'static + AsLogicalPlan, U: 'static + AsExecutionPlan> QueryBackend
}

async fn close_session(&self, session_id: &str) -> Result<()> {
self.state.session_manager.remove_session(session_id).await
self.remove_session_and_caches(session_id).await
}

async fn execute(
Expand Down
4 changes: 1 addition & 3 deletions ballista/scheduler/src/scheduler_server/grpc.rs
Original file line number Diff line number Diff line change
Expand Up @@ -444,9 +444,7 @@ impl<T: 'static + AsLogicalPlan, U: 'static + AsExecutionPlan> SchedulerGrpc
request: Request<RemoveSessionParams>,
) -> Result<Response<RemoveSessionResult>, Status> {
let session_params = request.into_inner();
self.state
.session_manager
.remove_session(&session_params.session_id)
self.remove_session_and_caches(&session_params.session_id)
.await
.map_err(|e| {
Status::internal(format!(
Expand Down
16 changes: 16 additions & 0 deletions ballista/scheduler/src/scheduler_server/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -349,6 +349,22 @@ impl<T: 'static + AsLogicalPlan, U: 'static + AsExecutionPlan> SchedulerServer<T
+ self.running_job_number()
}

/// Removes `session_id` and its `DataFrame::cache()` data. Without the
/// event loop the scheduler is stopping and the caches cannot be dropped:
/// the session is removed anyway, and the error returned.
pub(crate) async fn remove_session_and_caches(&self, session_id: &str) -> Result<()> {
match self.query_stage_event_loop.get_sender() {
Ok(sender) => self.state.remove_session(session_id, &sender).await,
Err(e) => {
self.state
.session_manager
.remove_session(session_id)
.await?;
Err(e)
}
}
}

/// Admits a job submission, or refuses it if the scheduler is shutting
/// down. [`submit_plan`](Self::submit_plan) puts the [`Admission`] in the
/// [`JobQueued`](QueryStageSchedulerEvent::JobQueued) event for the job, and
Expand Down
Loading
Loading