← Back to context

Comment by giancarlostoro

3 days ago

Lowkey I hate the "\" line continuation in Python to force PEP-8 compliance in a way... Is there any Pythonistas who would write the examples in there differently to achieve a similar level of readability?

> pipeline = task(get_data, branch=True) \

> | task(step1, workers=20) \

> | task(step2, workers=20) \

> | task(step3, workers=20, multiprocess=True)

8 comments

giancarlostoro

Reply
dec0dedab0de  3 days ago

I really don’t like overloading pipes like this. I would rather chain methods like how the django orm does it.

you could reassign every line, but it would look nicer with chained functions.

  pipeline = task(get_data, branch=True)
  pipeline = pipeline | task(step1, workers=20)
  pipeline = pipeline |  task(step2, workers=20)
  pipeline = pipeline |  task(step3, workers=20, multiprocess=True)

edit:

I would be tempted to do something like this:

  steps = [task(step1, workers=20),
           task(step2, workers=20),
           task(step3, workers=20, multiprocess=True)]
  pipeline = task(get_data, branch=True)

  for step in steps:
      pipeline =   pipeline.__or__(step)

  • Rickster35  3 days ago

    According to the docs, | is syntactic sugar for the .pipe method.

      pipeline = task(get_data, branch=True).pipe(
      task(step1, workers=20)).pipe(
      task(step2, workers=20)).pipe(
      task(step3, workers=20, multiprocess=True))

    That's probably the chained method approach for those with this preference.

  • me-vs-cat  1 day ago

    This style looks pretty good to me:

        pipeline = task(...)
    pipeline |= task(...)

    So does this style:

        steps = [task(...), task(...)]
    pipeline = functools.reduce(operator.or_, steps)

    But it appears you can just change "task" to "Task" and then:

        pipeline = pyper.Pipeline([Task(...), Task(...)])

morkalork  3 days ago

Wrap the statement in (...) and you can drop the backslashes. See also how people split up complicated loc queries in pandas on multiple lines too.

  • giancarlostoro  3 days ago

    Thank you! It seems this works for not just parens but square brackets and curly braces too! Only special requirement is indentation is consistent (duh though).

    I've not been doing Python day-to-day so I'm starting to lose my touch on all the nice little tricks.

    • Rickster35  3 days ago

      Yeah, this seems neatest if you don't like line breaks

      pipeline = (

          task(get_data, branch=True)

    | task(step1, workers=20)

    | task(step2, workers=20)

    | task(step3, workers=20, multiprocess=True)

      )

      Square brackets would create a list and braces would create a set of course. The contents still can be split over different lines-- just pointing that this syntax doesn't do the same thing.

      2 replies →