2 ( ProcPointSet, Status(..)
3 , callProcPoints, minimalProcPointSet
4 , addProcPointProtocols, splitAtProcPoints, procPointAnalysis
8 import Prelude hiding (zip, unzip, last)
12 import Cmm hiding (blockId)
18 import Data.List (sortBy)
21 import MkZipCfgCmm hiding (CmmBlock, CmmGraph, CmmTopZ)
30 import qualified Data.Map as Map
32 -- Compute a minimal set of proc points for a control-flow graph.
34 -- Determine a protocol for each proc point (which live variables will
35 -- be passed as arguments and which will be on the stack).
38 A proc point is a basic block that, after CPS transformation, will
39 start a new function. The entry block of the original function is a
40 proc point, as is the continuation of each function call.
41 A third kind of proc point arises if we want to avoid copying code.
42 Suppose we have code like the following:
45 if (...) { ..1..; call foo(); ..2..}
46 else { ..3..; call bar(); ..4..}
51 The statement 'x = y + z' can be reached from two different proc
52 points: the continuations of foo() and bar(). We would prefer not to
53 put a copy in each continuation; instead we would like 'x = y + z' to
54 be the start of a new procedure to which the continuations can jump:
57 if (...) { ..1..; push k_foo; jump foo_cps(); }
58 else { ..3..; push k_bar; jump bar_cps(); }
60 k_foo() { ..2..; jump k_join(y, z); }
61 k_bar() { ..4..; jump k_join(y, z); }
62 k_join(y, z) { x = y + z; return x; }
64 You might think then that a criterion to make a node a proc point is
65 that it is directly reached by two distinct proc points. (Note
66 [Direct reachability].) But this criterion is a bit too simple; for
67 example, 'return x' is also reached by two proc points, yet there is
68 no point in pulling it out of k_join. A good criterion would be to
69 say that a node should be made a proc point if it is reached by a set
70 of proc points that is different than its immediate dominator. NR
71 believes this criterion can be shown to produce a minimum set of proc
72 points, and given a dominator tree, the proc points can be chosen in
73 time linear in the number of blocks. Lacking a dominator analysis,
74 however, we turn instead to an iterative solution, starting with no
75 proc points and adding them according to these rules:
77 1. The entry block is a proc point.
78 2. The continuation of a call is a proc point.
79 3. A node is a proc point if it is directly reached by more proc
80 points than one of its predecessors.
82 Because we don't understand the problem very well, we apply rule 3 at
83 most once per iteration, then recompute the reachability information.
84 (See Note [No simple dataflow].) The choice of the new proc point is
85 arbitrary, and I don't know if the choice affects the final solution,
86 so I don't know if the number of proc points chosen is the
87 minimum---but the set will be minimal.
90 type ProcPointSet = BlockSet
93 = ReachedBy ProcPointSet -- set of proc points that directly reach the block
94 | ProcPoint -- this block is itself a proc point
96 instance Outputable Status where
98 | isEmptyBlockSet ps = text "<not-reached>"
99 | otherwise = text "reached by" <+>
100 (hsep $ punctuate comma $ map ppr $ blockSetToList ps)
101 ppr ProcPoint = text "<procpt>"
104 lattice :: DataflowLattice Status
105 lattice = DataflowLattice "direct proc-point reachability" unreached add_to False
106 where unreached = ReachedBy emptyBlockSet
107 add_to _ ProcPoint = noTx ProcPoint
108 add_to ProcPoint _ = aTx ProcPoint -- aTx because of previous case again
109 add_to (ReachedBy p) (ReachedBy p') =
110 let union = unionBlockSets p p'
111 in if sizeBlockSet union > sizeBlockSet p' then
112 aTx (ReachedBy union)
115 --------------------------------------------------
116 -- transfer equations
118 forward :: ForwardTransfers Middle Last Status
119 forward = ForwardTransfers first middle last exit
120 where first id ProcPoint = ReachedBy $ unitBlockSet id
123 last (LastCall _ (Just id) _ _ _) _ = LastOutFacts [(id, ProcPoint)]
124 last l x = LastOutFacts $ map (\id -> (id, x)) (succs l)
127 -- It is worth distinguishing two sets of proc points:
128 -- those that are induced by calls in the original graph
129 -- and those that are introduced because they're reachable from multiple proc points.
130 callProcPoints :: CmmGraph -> ProcPointSet
131 callProcPoints g = fold_blocks add (unitBlockSet (lg_entry g)) g
132 where add b set = case last $ unzip b of
133 LastOther (LastCall _ (Just k) _ _ _) -> extendBlockSet set k
136 minimalProcPointSet :: ProcPointSet -> CmmGraph -> FuelMonad ProcPointSet
137 -- Given the set of successors of calls (which must be proc-points)
138 -- figure ou the minimal set of necessary proc-points
139 minimalProcPointSet callProcPoints g = extendPPSet g (postorder_dfs g) callProcPoints
141 type PPFix = FuelMonad (ForwardFixedPoint Middle Last Status ())
143 procPointAnalysis :: ProcPointSet -> CmmGraph -> FuelMonad (BlockEnv Status)
144 -- Once you know what the proc-points are, figure out
145 -- what proc-points each block is reachable from
146 procPointAnalysis procPoints g =
147 let addPP env id = extendBlockEnv env id ProcPoint
148 initProcPoints = foldl addPP emptyBlockEnv (blockSetToList procPoints)
149 in liftM zdfFpFacts $
150 (zdfSolveFrom initProcPoints "proc-point reachability" lattice
151 forward (fact_bot lattice) $ graphOfLGraph g :: PPFix)
153 extendPPSet :: CmmGraph -> [CmmBlock] -> ProcPointSet -> FuelMonad ProcPointSet
154 extendPPSet g blocks procPoints =
155 do env <- procPointAnalysis procPoints g
156 let add block pps = let id = blockId block
157 in case lookupBlockEnv env id of
158 Just ProcPoint -> extendBlockSet pps id
160 procPoints' = fold_blocks add emptyBlockSet g
161 newPoints = mapMaybe ppSuccessor blocks
162 newPoint = listToMaybe newPoints
163 ppSuccessor b@(Block bid _) =
164 let nreached id = case lookupBlockEnv env id `orElse`
165 pprPanic "no ppt" (ppr id <+> ppr b) of
167 ReachedBy ps -> sizeBlockSet ps
168 block_procpoints = nreached bid
169 -- | Looking for a successor of b that is reached by
170 -- more proc points than b and is not already a proc
171 -- point. If found, it can become a proc point.
172 newId succ_id = not (elemBlockSet succ_id procPoints') &&
173 nreached succ_id > block_procpoints
174 in listToMaybe $ filter newId $ succs b
177 [] -> return procPoints'
178 pps -> extendPPSet g blocks
179 (foldl extendBlockSet procPoints' pps)
181 case newPoint of Just id ->
182 if elemBlockSet id procPoints' then panic "added old proc pt"
183 else extendPPSet g blocks (extendBlockSet procPoints' id)
184 Nothing -> return procPoints'
187 ------------------------------------------------------------------------
188 -- Computing Proc-Point Protocols --
189 ------------------------------------------------------------------------
193 There is one major trick, discovered by Michael Adams, which is that
194 we want to choose protocols in a way that enables us to optimize away
195 some continuations. The optimization is very much like branch-chain
196 elimination, except that it involves passing results as well as
197 control. The idea is that if a call's continuation k does nothing but
198 CopyIn its results and then goto proc point P, the call's continuation
199 may be changed to P, *provided* P's protocol is identical to the
200 protocol for the CopyIn. We choose protocols to make this so.
202 Here's an explanatory example; we begin with the source code (lines
203 separate basic blocks):
211 Zipperization converts this code as follows:
214 call g() returns to k;
221 What we'd like to do is assign P the same CopyIn protocol as k, so we
225 call g() returns to P;
227 P: CopyIn(x, y); ..2..;
229 Of course, P may be the target of more than one continuation, and
230 different continuations may have different protocols. Michael Adams
231 implemented a voting mechanism, but he thinks a simple greedy
232 algorithm would be just as good, so that's what we do.
236 data Protocol = Protocol Convention CmmFormals Area
238 instance Outputable Protocol where
239 ppr (Protocol c fs a) = text "Protocol" <+> ppr c <+> ppr fs <+> ppr a
241 -- | Function 'optimize_calls' chooses protocols only for those proc
242 -- points that are relevant to the optimization explained above.
243 -- The others are assigned by 'add_unassigned', which is not yet clever.
245 addProcPointProtocols :: ProcPointSet -> ProcPointSet -> CmmGraph -> FuelMonad CmmGraph
246 addProcPointProtocols callPPs procPoints g =
247 do liveness <- cmmLivenessZ g
248 (protos, g') <- optimize_calls liveness g
249 blocks'' <- add_CopyOuts protos procPoints g'
250 return $ LGraph (lg_entry g) blocks''
251 where optimize_calls liveness g = -- see Note [Separate Adams optimization]
252 do let (protos, blocks') =
253 fold_blocks maybe_add_call (init_protocols, emptyBlockEnv) g
254 protos' = add_unassigned liveness procPoints protos
255 blocks <- add_CopyIns callPPs protos' blocks'
256 let g' = LGraph (lg_entry g) (mkBlockEnv (map withKey (concat blocks)))
257 withKey b@(Block bid _) = (bid, b)
258 return (protos', runTx removeUnreachableBlocksZ g')
259 maybe_add_call :: CmmBlock -> (BlockEnv Protocol, BlockEnv CmmBlock)
260 -> (BlockEnv Protocol, BlockEnv CmmBlock)
261 -- ^ If the block is a call whose continuation goes to a proc point
262 -- whose protocol either matches the continuation's or is not yet set,
263 -- redirect the call (cf 'newblock') and set the protocol if necessary
264 maybe_add_call block (protos, blocks) =
265 case goto_end $ unzip block of
266 (h, LastOther (LastCall tgt (Just k) args res s))
267 | Just proto <- lookupBlockEnv protos k,
268 Just pee <- branchesToProcPoint k
269 -> let newblock = zipht h (tailOfLast (LastCall tgt (Just pee)
271 changed_blocks = insertBlock newblock blocks
272 unchanged_blocks = insertBlock block blocks
273 in case lookupBlockEnv protos pee of
274 Nothing -> (extendBlockEnv protos pee proto,changed_blocks)
276 if proto == proto' then (protos, changed_blocks)
277 else (protos, unchanged_blocks)
278 _ -> (protos, insertBlock block blocks)
280 branchesToProcPoint :: BlockId -> Maybe BlockId
281 -- ^ Tells whether the named block is just a branch to a proc point
282 branchesToProcPoint id =
283 let (Block _ t) = lookupBlockEnv (lg_blocks g) id `orElse`
284 panic "branch out of graph"
286 ZLast (LastOther (LastBranch pee))
287 | elemBlockSet pee procPoints -> Just pee
289 init_protocols = fold_blocks maybe_add_proto emptyBlockEnv g
290 maybe_add_proto :: CmmBlock -> BlockEnv Protocol -> BlockEnv Protocol
291 --maybe_add_proto (Block id (ZTail (CopyIn c _ fs _srt) _)) env =
292 -- extendBlockEnv env id (Protocol c fs $ toArea id fs)
293 maybe_add_proto _ env = env
294 -- JD: Is this proto stuff even necessary, now that we have
295 -- common blockification?
297 -- | For now, following a suggestion by Ben Lippmeier, we pass all
298 -- live variables as arguments, hoping that a clever register
299 -- allocator might help.
301 add_unassigned :: BlockEnv CmmLive -> ProcPointSet -> BlockEnv Protocol ->
303 add_unassigned = pass_live_vars_as_args
305 pass_live_vars_as_args :: BlockEnv CmmLive -> ProcPointSet ->
306 BlockEnv Protocol -> BlockEnv Protocol
307 pass_live_vars_as_args _liveness procPoints protos = protos'
308 where protos' = foldBlockSet addLiveVars protos procPoints
309 addLiveVars :: BlockId -> BlockEnv Protocol -> BlockEnv Protocol
310 addLiveVars id protos =
311 case lookupBlockEnv protos id of
313 Nothing -> let live = emptyRegSet
314 --lookupBlockEnv _liveness id `orElse`
315 --panic ("no liveness at block " ++ show id)
316 formals = uniqSetToList live
317 prot = Protocol Private formals $ CallArea $ Young id
318 in extendBlockEnv protos id prot
321 -- | Add copy-in instructions to each proc point that did not arise from a call
322 -- instruction. (Proc-points that arise from calls already have their copy-in instructions.)
324 add_CopyIns :: ProcPointSet -> BlockEnv Protocol -> BlockEnv CmmBlock ->
325 FuelMonad [[CmmBlock]]
326 add_CopyIns callPPs protos blocks =
327 liftUniq $ mapM maybe_insert_CopyIns (blockEnvToList blocks)
328 where maybe_insert_CopyIns (_, b@(Block id t))
329 | not $ elemBlockSet id callPPs
330 = case lookupBlockEnv protos id of
331 Just (Protocol c fs _area) ->
332 do LGraph _ blocks <-
333 lgraphOfAGraph (mkLabel id <*> copyInSlot c fs <*> mkZTail t)
334 return (map snd $ blockEnvToList blocks)
335 Nothing -> return [b]
336 | otherwise = return [b]
338 -- | Add a CopyOut node before each procpoint.
339 -- If the predecessor is a call, then the copy outs should already be done by the callee.
340 -- Note: If we need to add copy-out instructions, they may require stack space,
341 -- so we accumulate a map from the successors to the necessary stack space,
342 -- then update the successors after we have finished inserting the copy-outs.
344 add_CopyOuts :: BlockEnv Protocol -> ProcPointSet -> CmmGraph ->
345 FuelMonad (BlockEnv CmmBlock)
346 add_CopyOuts protos procPoints g = fold_blocks mb_copy_out (return emptyBlockEnv) g
347 where mb_copy_out :: CmmBlock -> FuelMonad (BlockEnv CmmBlock) ->
348 FuelMonad (BlockEnv CmmBlock)
349 mb_copy_out b@(Block bid _) z | bid == lg_entry g = skip b z
351 case last $ unzip b of
352 LastOther (LastCall _ _ _ _ _) -> skip b z -- copy out done by callee
354 copy_out b z = fold_succs trySucc b init >>= finish
355 where init = z >>= (\bmap -> return (b, bmap))
357 if elemBlockSet succId procPoints then
358 case lookupBlockEnv protos succId of
360 Just (Protocol c fs _area) -> insert z succId $ copyOutSlot c fs
364 (b, bs) <- insertBetween b m succId
365 -- pprTrace "insert for succ" (ppr succId <> ppr m) $ do
366 return $ (b, foldl (flip insertBlock) bmap bs)
367 finish (b@(Block bid _), bmap) =
368 return $ (extendBlockEnv bmap bid b)
369 skip b@(Block bid _) bs =
370 bs >>= (\bmap -> return (extendBlockEnv bmap bid b))
372 -- At this point, we have found a set of procpoints, each of which should be
373 -- the entry point of a procedure.
374 -- Now, we create the procedure for each proc point,
375 -- which requires that we:
376 -- 1. build a map from proc points to the blocks reachable from the proc point
377 -- 2. turn each branch to a proc point into a jump
378 -- 3. turn calls and returns into jumps
379 -- 4. build info tables for the procedures -- and update the info table for
380 -- the SRTs in the entry procedure as well.
381 -- Input invariant: A block should only be reachable from a single ProcPoint.
382 splitAtProcPoints :: CLabel -> ProcPointSet-> ProcPointSet -> BlockEnv Status ->
383 CmmTopZ -> FuelMonad [CmmTopZ]
384 splitAtProcPoints entry_label callPPs procPoints procMap
385 (CmmProc (CmmInfo gc upd_fr info_tbl) top_l top_args
386 (stackInfo, g@(LGraph entry blocks))) =
387 do -- Build a map from procpoints to the blocks they reach
388 let addBlock b@(Block bid _) graphEnv =
389 case lookupBlockEnv procMap bid of
390 Just ProcPoint -> add graphEnv bid bid b
391 Just (ReachedBy set) ->
392 case blockSetToList set of
394 [id] -> add graphEnv id bid b
395 _ -> panic "Each block should be reachable from only one ProcPoint"
396 Nothing -> pprPanic "block not reached by a proc point?" (ppr bid)
397 add graphEnv procId bid b = extendBlockEnv graphEnv procId graph'
398 where graph = lookupBlockEnv graphEnv procId `orElse` emptyBlockEnv
399 graph' = extendBlockEnv graph bid b
400 graphEnv <- return $ fold_blocks addBlock emptyBlockEnv g
401 -- Build a map from proc point BlockId to labels for their new procedures
402 -- Due to common blockification, we may overestimate the set of procpoints.
403 let add_label map pp = return $ Map.insert pp lbl map
404 where lbl = if pp == entry then entry_label else blockLbl pp
405 procLabels <- foldM add_label Map.empty
406 (filter (elemBlockEnv blocks) (blockSetToList procPoints))
407 -- For each procpoint, we need to know the SP offset on entry.
408 -- If the procpoint is:
409 -- - continuation of a call, the SP offset is in the call
410 -- - otherwise, 0 -- no overflow for passing those variables
411 let add_sp_off b env =
412 case last (unzip b) of
413 LastOther (LastCall {cml_cont = Just succ, cml_ret_args = off,
414 cml_ret_off = updfr_off}) ->
415 extendBlockEnv env succ (off, updfr_off)
417 spEntryMap = fold_blocks add_sp_off (mkBlockEnv [(entry, stackInfo)]) g
418 getStackInfo id = lookupBlockEnv spEntryMap id `orElse` (0, Nothing)
419 -- In each new graph, add blocks jumping off to the new procedures,
420 -- and replace branches to procpoints with branches to the jump-off blocks
421 let add_jump_block (env, bs) (pp, l) =
422 do bid <- liftM mkBlockId getUniqueM
423 let b = Block bid (ZLast (LastOther jump))
424 (argSpace, _) = getStackInfo pp
425 jump = LastCall (CmmLit (CmmLabel l')) Nothing argSpace 0 Nothing
426 l' = if elemBlockSet pp callPPs then entryLblToInfoLbl l else l
427 return (extendBlockEnv env pp bid, b : bs)
428 add_jumps (newGraphEnv) (ppId, blockEnv) =
429 do let needed_jumps = -- find which procpoints we currently branch to
430 foldBlockEnv' add_if_branch_to_pp [] blockEnv
431 add_if_branch_to_pp block rst =
432 case last (unzip block) of
433 LastOther (LastBranch id) -> add_if_pp id rst
434 LastOther (LastCondBranch _ ti fi) ->
435 add_if_pp ti (add_if_pp fi rst)
436 LastOther (LastSwitch _ tbl) -> foldr add_if_pp rst (catMaybes tbl)
438 add_if_pp id rst = case Map.lookup id procLabels of
439 Just x -> (id, x) : rst
441 (jumpEnv, jumpBlocks) <-
442 foldM add_jump_block (emptyBlockEnv, []) needed_jumps
443 -- update the entry block
444 let b = expectJust "block in env" $ lookupBlockEnv blockEnv ppId
445 off = getStackInfo ppId
446 blockEnv' = extendBlockEnv blockEnv ppId b
447 -- replace branches to procpoints with branches to jumps
448 LGraph _ blockEnv'' = replaceBranches jumpEnv $ LGraph ppId blockEnv'
449 -- add the jump blocks to the graph
450 blockEnv''' = foldl (flip insertBlock) blockEnv'' jumpBlocks
451 let g' = (off, LGraph ppId blockEnv''')
452 -- pprTrace "g' pre jumps" (ppr g') $ do
453 return (extendBlockEnv newGraphEnv ppId g')
454 graphEnv <- foldM add_jumps emptyBlockEnv $ blockEnvToList graphEnv
455 let to_proc (bid, g) | elemBlockSet bid callPPs =
457 CmmProc (CmmInfo gc upd_fr info_tbl) top_l top_args (replacePPIds g)
459 CmmProc emptyContInfoTable lbl [] (replacePPIds g)
460 where lbl = expectJust "pp label" $ Map.lookup bid procLabels
462 CmmProc (CmmInfo Nothing Nothing CmmNonInfoTable) lbl [] (replacePPIds g)
463 where lbl = expectJust "pp label" $ Map.lookup bid procLabels
464 -- References to procpoint IDs can now be replaced with the infotable's label
465 replacePPIds (x, g) = (x, map_nodes id (mapExpMiddle repl) (mapExpLast repl) g)
466 where repl e@(CmmLit (CmmBlock bid)) =
467 case Map.lookup bid procLabels of
468 Just l -> CmmLit (CmmLabel (entryLblToInfoLbl l))
471 -- The C back end expects to see return continuations before the call sites.
472 -- Here, we sort them in reverse order -- it gets reversed later.
473 let (_, block_order) = foldl add_block_num (0::Int, emptyBlockEnv) (postorder_dfs g)
474 add_block_num (i, map) (Block bid _) = (i+1, extendBlockEnv map bid i)
475 sort_fn (bid, _) (bid', _) =
476 compare (expectJust "block_order" $ lookupBlockEnv block_order bid)
477 (expectJust "block_order" $ lookupBlockEnv block_order bid')
478 procs <- return $ map to_proc $ sortBy sort_fn $ blockEnvToList graphEnv
479 return -- pprTrace "procLabels" (ppr procLabels)
480 -- pprTrace "splitting graphs" (ppr procs)
482 splitAtProcPoints _ _ _ _ t@(CmmData _ _) = return [t]
484 ----------------------------------------------------------------
487 Note [Direct reachability]
489 Block B is directly reachable from proc point P iff control can flow
490 from P to B without passing through an intervening proc point.
493 ----------------------------------------------------------------
496 Note [No simple dataflow]
498 Sadly, it seems impossible to compute the proc points using a single
499 dataflow pass. One might attempt to use this simple lattice:
501 data Location = Unknown
502 | InProc BlockId -- node is in procedure headed by the named proc point
503 | ProcPoint -- node is itself a proc point
505 At a join, a node in two different blocks becomes a proc point.
506 The difficulty is that the change of information during iterative
507 computation may promote a node prematurely. Here's a program that
508 illustrates the difficulty:
517 L2: if (...) { g(); goto L1; }
521 The only proc-point needed (besides the entry) is L1. But in an
522 iterative analysis, consider what happens to L2. On the first pass
523 through, it rises from Unknown to 'InProc entry', but when L1 is
524 promoted to a proc point (because it's the successor of g()), L1's
525 successors will be promoted to 'InProc L1'. The problem hits when the
526 new fact 'InProc L1' flows into L2 which is already bound to 'InProc entry'.
527 The join operation makes it a proc point when in fact it needn't be,
528 because its immediate dominator L1 is already a proc point and there
529 are no other proc points that directly reach L2.
534 {- Note [Separate Adams optimization]
535 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
536 It may be worthwhile to attempt the Adams optimization by rewriting
537 the graph before the assignment of proc-point protocols. Here are a
540 g() returns to k; g() returns to L;
541 k: CopyIn c ress; goto L:
543 L: // no CopyIn node here L: CopyIn c ress;
546 And when c == c' and ress == ress', this also:
548 g() returns to k; g() returns to L;
549 k: CopyIn c ress; goto L:
551 L: CopyIn c' ress' L: CopyIn c' ress' ;
553 In both cases the goal is to eliminate k.