Alignment is Real // Shiva Bhattacharjee // #260
MLOps.community
Deep Dive
Why did TrueLaw choose DSPy over other tools like LangChain?
TrueLaw chose DSPy due to its modular nature, ease of customization, and better object hierarchy, which allowed for more efficient and transparent query rewriting and iterative improvements.
Why is fine-tuning necessary for domain-specific tasks in legal AI?
Fine-tuning is necessary for domain-specific tasks in legal AI because off-the-shelf models often lack the precision and quality required by lawyers, who are not skilled prompt engineers. Fine-tuning helps contextualize queries and align the output with the firm's specific expectations.
Why is embedding model fine-tuning cost-effective for legal AI tasks?
Embedding model fine-tuning is cost-effective because the models are relatively small and the main cost is generating contrastive data, which is cheaper compared to training large models from scratch.
Why did TrueLaw decide to use SaaS providers for infrastructure rather than building it in-house?
TrueLaw decided to use SaaS providers for infrastructure to leverage existing services, reduce costs, and focus on their core IP, which is data generation and fine-tuning. This approach is more efficient and scalable, especially for a startup with resource constraints.
Why did TrueLaw choose to use Temporal for managing long-running workflows?
TrueLaw chose Temporal for managing long-running workflows because it provided a robust and flexible workflow engine that handled retries, interruptions, and notifications, which would have taken significant time and effort to build in-house.
Why does Shiva believe his broad experience across different tech stacks has been beneficial?
Shiva believes his broad experience has been beneficial because it allows him to draw parallels between seemingly unrelated areas, apply fundamental principles across different domains, and understand the core concepts that are universally applicable in solving performance and system-level issues.
- DSPy is used for its modularity and iterative improvement capabilities.
- Prompt engineering in DSPy is more adaptable than in Langchain.
- DSPy's modular design allows for easy integration of custom re-rankers.
Shownotes Transcript