Comment by naddeo

1 year ago

I was surprised to see the example in the blog. Python actually has come pretty far with types but the blog's example doesn't really highlight it. For structural static typing, something like this is nicer as an example

    from typing import Protocol, Tuple, TypedDict


    class Foo(TypedDict):
        foo: str
        bar: int
        baz: Tuple[str, int]
        baaz: Tuple[float, ...]


    class Functionality(Protocol):
        def do(self, it: Foo): ...


    class MyFunctionality: # not explicitly implemented
        def do(self, it: Foo): ...


    class DoIt:
        def execute(self, it: Foo, func: Functionality): ...


    doit = DoIt().execute({ # Type checks
        "foo": "foo",
        "bar": 7, 
        "baz": ("str", 2), 
        "baaz": (1.0, 2.0)}, MyFunctionality())

Protocols and TypedDicts let you do nice structural stuff, similar typescript (though not as feature complete). Types are good enough on python that I would never consider a project without them, and I work with pandas and numpy a lot. You change your workflow a little bit so that you end up quarantining the code that interfaces with third-party libraries that don't have good type support behind your own functions that do. There are other pretty cool type things as well. Python is definitely in much better shape than it was.

Combine all of that with Pyright's ability to do more advanced type inference and its like a whole new language experience.

13 comments

naddeo

bormaj 1 year ago

I would supplement this by suggesting `pydantic` models instead of `TypedDict`s. This library has become a core utility for me as it greatly improves the developer experience with typing/validation/serialization support.

maleldil 1 year ago
They fulfil different roles. pydantic models would be an alternative to dataclasses, attrs or data-centric classes in general. TypedDict is used when you're stuck with dicts and can't convert them to a specific class.
- catlover76 1 year ago
  
  [dead]
naddeo 1 year ago

Yeah they're kind of different. I'm only really talking about TypedDicts because the original post was related to structural typing, which isn't what pydantic does. I do reach for pydantic first myself.

oivey 1 year ago

TypedDicts are a really disappointing feature. Typing fails if you pass it a dictionary with extra keys, so you can’t use it for many structural typing use cases.

It’s even more disappointing because this isn’t just an oversight. The authors have deliberately made having additional keys an error. Apparently, this even a divergence from how TypeScript checks dictionaries.

Syntaf 1 year ago

For what it's worth, TypedDict was a bit ahead of it's own time. Python 3.12 is really the turning point for being able to leverage them effectively for stuff like **kwargs[1][2]

    from typing import TypedDict, Unpack, NotRequired

    class Movie(TypedDict):
        name: str
        year: NotRequired[int]

    def foo(**kwargs: Unpack[Movie]) -> None: ...

[1] https://typing.readthedocs.io/en/latest/spec/callables.html#... [2] https://peps.python.org/pep-0692/

o11c 1 year ago

How does that work for partial forwarding? Back when I last checked there was no imagining a DRY solution for that even for linear cases, let alone diamond inheritance.

  # Simple case, any halfway-decent type system should be able to handle this.
  def inner(*, i): pass
  def middle(*, m, **kwargs): inner(**kwargs)
  def outer(*, o, **kwargs): middle(**kwargs)
  kwargs = ...; outer(**kwargs)

  # More complicated case, but fairly common in Python code.
  class A:
    def __init__(self, *, a):
      pass
  class B(A):
    def __init__(self, *, b, **kwargs):
      super().__init__(**kwargs)
  class C(A):
    def __init__(self, *, c, **kwargs):
      super().__init__(**kwargs)
  class D(B, C):
    def __init__(self, *, d, **kwargs):
      super().__init__(**kwargs)
  kwargs = ...; D(**kwargs)

  # An even more complicated case involves `kwargs.pop()`, forwarding without `**`.

Can the above be typechecked yet?

o11c 1 year ago
To be fair, that behavior on typescript's part is a major hole for bugs to slip through.
Specifically: absent optional keys + extra keys is fundamentally indistinguishable from miseptl keys.
- chuckadams 1 year ago
  
  `interface` in TS allows extra keys, `type` does do not. Usually best to use `type` and add an intersection with some other type if you want extras (`Record<string,unknown>` is all right for arbitrary extra keys)
- oivey 1 year ago
  
  The current situation is worse, IMO. Lots of code that could be checked cannot, and lots of code that could have a more clearly defined interface does not.
arwineap 1 year ago
I'm not sure I understand.
TypedDicts are disappointing because you can't partially define a type? That seems like a success
In go you would need to define all fields in your struct and if you needed unstructured data you would have to define a map, which even then is partially typed
How should extra keys behave in "typed python?"
- oivey 1 year ago
  
  Go is a terrible point of comparison. Python’s type system should not pointlessly match what other languages have chosen due to some myopic dogma.
  Python is a dynamic language where one of its major features is structural subtyping, aka duck typing. It’s effectively an alternative to inheritance. Features have been added to help support this, like TypedDicts and Protocols already. They don’t go far enough.
  I want to be able to say “this function argument is a dictionary that has at least the keys a, b, and c.” It gives the contract that the function only accesses those keys, and others will be ignored. The type checker can check that the function doesn’t access undeclared keys and the annotation helps communicate to client code what the interface is supposed to be.
  Lots of Python’s bolted on type checking seems to be straight jacketed to match what’s in C, Java, Go, etc. Those languages don’t contain the only possible type systems or static checkers. There’s a serious lack of imagination. Python’s type system should be designed around enabling and making safer what the language is already good at.
  
  1 reply →