Teradata connector-Spark-Scala(sink/source)
If there is a usecase to add Teradata as a connector to a Spark application (written in scala), then here you can refer below code snippet(Test.scala) which will help you to read from Teradata and writing to this locally (provided you have been given with connection details)
- Try test connection from DB client as mentioned below.
2. First you have to create a sample table in Teradata and import data to it from any file system(say csv). You can create table wither as SET/MULTISET.
3. Next you have to read this table as a Dataframe to your Spark.
4. After reading you can do some query onto it and create a new table where you can write the results of the query as a dataframe there into Teradata.
5. And in build.sbt, add the below library.(it depends on Scala version used in your application).
6. You have to check official Teradata site and download 2 jars to your local system.
a. tdgssconfig.jar and b. terajdbc4.jar.
7. Add those 2 jars as shown below in your intelliJ/File/ProjectSettings/Modules/Dependencies/+ icon for attaching and then apply and OK. Build you application and execute Test case.
8. Create a Test.scala file, run it locally using the above and verify if you are able to connect to Teradata and do read/write operations.
This is written to do a quick connection test from your local environement to Teradata. Hope this helps!